{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SVM on Pima Indians Diabetes Data Set\n",
    "\n",
    "数据说明：\n",
    "Pima Indians Diabetes Data Set（皮马印第安人糖尿病数据集） 根据现有的医疗信息预测5年内皮马印第安人糖尿病发作的概率。   \n",
    "\n",
    "数据集共9个字段: \n",
    "0列为怀孕次数；\n",
    "1列为口服葡萄糖耐量试验中2小时后的血浆葡萄糖浓度；\n",
    "2列为舒张压（单位:mm Hg）\n",
    "3列为三头肌皮褶厚度（单位：mm）\n",
    "4列为餐后血清胰岛素（单位:mm）\n",
    "5列为体重指数（体重（公斤）/ 身高（米）^2）\n",
    "6列为糖尿病家系作用\n",
    "7列为年龄\n",
    "8列为分类变量（0或1）\n",
    "\n",
    "数据链接：https://archive.ics.uci.edu/ml/datasets/Pima+Indians+Diabetes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 首先 import 必要的模块\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "#SVM并不能直接输出各类的概率，所以在这个例子中我们用正确率作为模型预测性能的度量\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#input data\n",
    "train = pd.read_csv(\"FE_pima-indians-diabetes.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "#  get labels\n",
    "y_train = train['Target']   \n",
    "X_train = train.drop([\"Target\"], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型训练\n",
    "由于速度较慢，这里只调整了超参数C，没调L1/L2正则函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7669382904677022\n",
      "{'C': 0.01}\n"
     ]
    }
   ],
   "source": [
    "#线性SVM\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "\n",
    "Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "param_grid = {'C': Cs}\n",
    "grid = GridSearchCV(SVC(kernel='linear'), param_grid, cv=5, n_jobs = 4)\n",
    "\n",
    "grid.fit(X_train, y_train)\n",
    "print(grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.6497496  0.76693829 0.76564808 0.76434938 0.76434938 0.76434938\n",
      " 0.76565657]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dfXRc9X3n8fdHT5Zt+fkhgG2QbUzCQwhuDNgQ0pIWAt0UmnaX4ITTZNsNySak3fQkG9gGkpBNTzfZTXLakpySJidNSaCEPJQWdzEbaGkrGWyCebAdwCPHWJgHeWT52ZY0+u4fcwVjeSyPZI3uzOjzOkeHuXd+9+p7sTQf3d/v3vtTRGBmZjZUXdoFmJlZZXJAmJlZUQ4IMzMrygFhZmZFOSDMzKyohrQLGCtz586N1tbWtMswM6sqTzzxxK6ImFfsvZoJiNbWVjZs2JB2GWZmVUXS9uO95y4mMzMrygFhZmZFOSDMzKyomhmDKKavr4/Ozk4OHz6cdiljrrm5mYULF9LY2Jh2KWZWo2o6IDo7O5k2bRqtra1ISrucMRMRZLNZOjs7Wbx4cdrlmFmNqukupsOHDzNnzpyaCgcAScyZM6cmz4zMrHLUdEAANRcOg2r1uMysctR0F9NEs+dQH3et2059nZjSVM/kxnqmNDXkXzfVMyX5mtzUwJTG/LpJDXUOGzMrygFRZj09PfzgBz/gYx/72Ii3/frXv86NN97IlClTSmr/ww07+MqDz43oe9QJpjQ1vB4g+VDJh8fkxoaCUBkMmIaj2hwTQI1H76uuzuFjVq0cEGXW09PDN77xjVEHxA033FByQLRlsiyeO5UH/vAdHOzNcag3x8HeHAd7+9943ZfjUG9/sj7H4b7c668H1x9K1nUfOPTGumTb3MDIJphqbqw7KlQKg2VyUz1TGgvOaoaE0bEBVRBGjfU01Nd8D6kZAH25gdd/l4v9bk+d1MA7zyr6tIyT4oAos5tvvplMJsMFF1zAFVdcwfz587n33ns5cuQI733ve/nCF77AgQMHuO666+js7CSXy3Hrrbfy6quvsnPnTi6//HLmzp3LI488Muz36csN8FhHlt9eviD5IB37f9qIoDc3UPDDOfiD2p8Ez9FB80bYHP1Dfag3x2v7Dh+zrjc3MKJ6murr3giTImcwU4qFTtK9dqIzoKYGh4+NTG9/8rvRV/xD/EAJvxtFA6AvR19u+D/M3rZopgPiZHzhHzaxeefeMd3nOadN53O/de6wbf7sz/6MZ599lo0bN7J27Vruu+8+Hn/8cSKCa665hkcffZSuri5OO+00HnjgAQD27NnDjBkz+OpXv8ojjzzC3LlzT1jLMy/t4UBvjlVL54zJsRUjiUkN9UxqqGdmaSc1I9KfGzgqaI468+nNceiYX7yCX7i+N8Kq52AvO3ve+CU8cKSfI/0jC5+GOg3TrVbP1MJuucIgOs64T0OdA6fS5CLyH9xHSvgD58ixZ+BDP9T7T+LseuqkN/54OWV649E/e031TD3Oz9lguxmTy3M/1IQJiEqwdu1a1q5dy/LlywHYv38/L7zwApdddhmf+tSn+MxnPsN73vMeLrvsshHvuz2TBWDlkvIFRLk11Ncxvb6O6c1j/8OeG4jXf9mLnQEdKtLVVti1dqi3nwNHcuw93M+re4ec/fTlxrxeqwylfohPGeaPhclN+W2rcXxuwgTEif7SHw8RwS233MJHPvKRY9574oknWLNmDbfccgtXXnklt91224j23Z7J8pZTpjG3ZdJYlVtT6utEy6QGWiaN/Y/8wEBwuD93bNdAwbhPbmBkZzBWfnVSTXyIl9OECYi0TJs2jX379gHw7ne/m1tvvZUPfOADtLS08NJLL9HY2Eh/fz+zZ8/mhhtuoKWlhe9+97tHbXuiLqYj/TnW/7Kb1RedXu7DsSLq6lS2cR+zNPknuszmzJnDpZdeynnnncfVV1/N+9//flatWgVAS0sLd911F1u3buXTn/40dXV1NDY28s1vfhOAG2+8kauvvppTTz112EHqJ1/s4Uj/AJeUcfzBzCYeRYxsYKVSrVixIoZOGLRlyxbOPvvslCoqv8Hj+9pDz/MXD7/Ak7ddWbbBKjOrTZKeiIgVxd4r66UVkq6S9JykrZJuLvL+1yRtTL6el9RT8N7pktZK2iJps6TWctZazdozWc5bMMPhYGZjqmxdTJLqgTuAK4BOYL2k+yNi82CbiPhkQftPAMsLdvE94EsR8ZCkFsCjfEUc6s3x5I7d/P6lfqqrmY2tcp5BXARsjYiOiOgF7gGuHab9auBuAEnnAA0R8RBAROyPiIOjKaJWutCGGjyuDdu76ctFWe9/MLOJqZwBsQDYUbDcmaw7hqQzgMXAw8mqs4AeST+W9KSkryRnJEO3u1HSBkkburq6jtlvc3Mz2Wy25kJicD6I5uZm2jJZGurEha2z0y7LzGpMOa9iKnYB8fE+qa8H7ouIwTuOGoDLyHc5vQj8HfAh4NtH7SziTuBOyA9SD93pwoUL6ezspFh4VLvBGeXa1jzO2xbNZGoZru83s4mtnJ8qncCiguWFwM7jtL0e+PiQbZ+MiA4AST8FVjIkIE6ksbGxpmdc23u4j2c6e/j45WemXYqZ1aBydjGtB5ZJWiypiXwI3D+0kaQ3A7OA9iHbzpI0+PSpdwGbh2470a3f1s1A4PEHMyuLsgVERPQDNwEPAluAeyNik6TbJV1T0HQ1cE8UDBQkXU2fAn4m6Rny3VXfKlet1aotk6WpoY5fOX1W2qWYWQ0qa8d1RKwB1gxZd9uQ5c8fZ9uHgPPLVlwNaM9kefvps2huPGb83szspPkZxFVq94FeNr+814/XMLOycUBUqXUd+cd7X3KmA8LMysMBUaXaMlmmNNVz/sKZaZdiZjXKAVGl2juyXNg6m0bPy2xmZeJPlyr02t7DbH1tv8cfzKysHBBVqD0Zf/D9D2ZWTg6IKtSeyTKtuYFzT5uRdilmVsMcEFWoLZNl5ZI51E/w+XLNrLwcEFWmc/dBXuw+yKol7l4ys/JyQFSZ9ozvfzCz8eGAqDLtmSxzpjZx1vxpaZdiZjXOAVFFIiI//rB0DnUefzCzMnNAVJFtuw7wyt7DHn8ws3HhgKgig/c/+AY5MxsPDogq0pbJcsr0ZhbPnZp2KWY2ATggqkREsC6TZdXSOUgefzCz8nNAVInnX91P9kCvH69hZuPGAVEl2jK7AI8/mNn4cUBUibZMlkWzJ7Nw1pS0SzGzCcIBUQVyA8FjHVkuWTI37VLMbAJxQFSBzTv3svdwvx+vYWbjygFRBQbHH3yDnJmNJwdEFWjvyLJ03lTmT29OuxQzm0AcEBWuLzfA49u6uWSpxx/MbHw5ICrc0509HOzN+fJWMxt3ZQ0ISVdJek7SVkk3F3n/a5I2Jl/PS+oZ8v50SS9J+sty1lnJ2rbmn790sccfzGycNZRrx5LqgTuAK4BOYL2k+yNi82CbiPhkQftPAMuH7OaLwL+Uq8Zq0N6R5exTpzN7alPapZjZBFPOM4iLgK0R0RERvcA9wLXDtF8N3D24IOntwJuAtWWssaId7suxYftudy+ZWSrKGRALgB0Fy53JumNIOgNYDDycLNcB/wf49HDfQNKNkjZI2tDV1TUmRVeSn7+4m97+AV/eamapKGdAFHvkaByn7fXAfRGRS5Y/BqyJiB3HaZ/fWcSdEbEiIlbMmzfvJEqtTOsyWeoEFy2ZnXYpZjYBlW0MgvwZw6KC5YXAzuO0vR74eMHyKuAySR8DWoAmSfsj4piB7lrWlsny1oUzmd7cmHYpZjYBlTMg1gPLJC0GXiIfAu8f2kjSm4FZQPvguoj4QMH7HwJWTLRwOHCkn407evjwO5ekXYqZTVBl62KKiH7gJuBBYAtwb0RsknS7pGsKmq4G7omI43U/TUgbtu+mfyA8/mBmqSnnGQQRsQZYM2TdbUOWP3+CfXwX+O4Yl1bx2jK7aKwXK1pnpV2KmU1QvpO6QrVnsixfNIspTWXNcDOz43JAVKA9h/p49qU9rPT9D2aWIgdEBXp8WzcD4elFzSxdDogK1JbZxaSGOpafPjPtUsxsAnNAVKD2TJYVrbOY1FCfdilmNoE5ICpMdv8RfvHKPs//YGapc0BUmHUd3QCs8viDmaXMAVFh2jK7aJnUwPkLZqRdiplNcA6ICtPekeXC1lk01PufxszS5U+hCvLKnsN0dB3w+IOZVQQHRAVp79gFePzBzCqDA6KCtG3NMmNyI+ecOj3tUszMHBCVpL0jy8ols6mrKzbXkpnZ+HJAVIgd3Qfp3H3I4w9mVjEcEBWiLePxBzOrLA6ICtGeyTK3pYll81vSLsXMDHBAVISIoC2TZdXSuUgefzCzyuCAqACZrgO8tu+IH+9tZhXFAVEB2juyAJ5/2swqigOiArRndnHajGbOmDMl7VLMzF7ngEjZwEDQ7vEHM6tADoiUPffqPnYf7PPlrWZWcRwQKWvLJOMPDggzqzAOiJS1Z3bROmcKC2ZOTrsUM7OjlDUgJF0l6TlJWyXdXOT9r0namHw9L6knWX+BpHZJmyQ9Lel95awzLf25AR7r6PbZg5lVpIZy7VhSPXAHcAXQCayXdH9EbB5sExGfLGj/CWB5sngQ+L2IeEHSacATkh6MiJ5y1ZuGTTv3su9IP6v8/CUzq0DlPIO4CNgaER0R0QvcA1w7TPvVwN0AEfF8RLyQvN4JvAbMK2OtqXh9/MH3P5hZBTphQEi6SdKsUex7AbCjYLkzWVfse5wBLAYeLvLeRUATkCny3o2SNkja0NXVNYoS09WW2cVZb2ph3rRJaZdiZnaMUs4gTiHfPXRvMqZQ6sX6xdrFcdpeD9wXEbmjdiCdCvwt8J8jYuCYnUXcGRErImLFvHnVdYLR2z/Ahl/u9tmDmVWsEwZERHwWWAZ8G/gQ8IKkP5W09ASbdgKLCpYXAjuP0/Z6ku6lQZKmAw8An42IdSeqs9o81dnDob6cxx/MrGKVNAYREQG8knz1A7OA+yR9eZjN1gPLJC2W1EQ+BO4f2kjSm5P9tResawJ+AnwvIn5Y4rFUlbatWSRYuWR22qWYmRVVyhjEH0p6Avgy8O/AWyPivwJvB373eNtFRD9wE/AgsAW4NyI2Sbpd0jUFTVcD9yQhNOg64J3Ahwoug71gpAdXydo7dnHOqdOZOaUp7VLMzIoq5TLXucDvRMT2wpURMSDpPcNtGBFrgDVD1t02ZPnzRba7C7irhNqq0uG+HD/f3sMHLzkj7VLMzI6rlC6mNUD34IKkaZIuBoiILeUqrJY9sX03vbkBzz9tZhWtlID4JrC/YPlAss5GqS2zi/o6ceFijz+YWeUqJSBUOD6QXG5atjuwJ4L2TJbzF86gZZL/N5pZ5SolIDqSgerG5OuPgI5yF1ar9h/p56nOPZ5e1MwqXikB8VHgEuAl8vc2XAzcWM6iatn6bd3kBsLjD2ZW8U7YxxERr5G/h8HGQHtHlqb6Ot5+xmieXmJmNn5OGBCSmoE/AM4FmgfXR8Tvl7GumtWW2cXy02fS3FifdilmZsMqpYvpb8k/j+ndwL+Qf2TGvnIWVat6Dvayaededy+ZWVUoJSDOjIhbgQMR8TfAfwDeWt6yatNj27qJ8PSiZlYdSgmIvuS/PZLOA2YArWWrqIa1Z7I0N9ZxwaKZaZdiZnZCpVyIf2cyH8RnyT9srwW4taxV1ai2zC4ubJ1NU4OnAjezyjdsQEiqA/ZGxG7gUWDJuFRVg7r2HeH5V/fz28uLzplkZlZxhv1TNrlr+qZxqqWmrevITy/qAWozqxal9HU8JOlTkhZJmj34VfbKakxbJsu0SQ2cd9r0tEsxMytJKWMQg/c7fLxgXeDuphFpz+zi4iWzaaj3+IOZVYdS7qRePB6F1LKdPYf4ZfYgN6z0/A9mVj1KuZP694qtj4jvjX05tak94/EHM6s+pXQxXVjwuhn4deDngAOiRG2ZLLOmNPKWU6alXYqZWclK6WL6ROGypBnkH79hJYgI1nVkWblkDnV1SrscM7OSjWbE9CCwbKwLqVUvdh/kpZ5Dnv/BzKpOKWMQ/0D+qiXIB8o5wL3lLKqWtCXjD6s8/mBmVaaUMYj/XfC6H9geEZ1lqqfmtGeyzJs2iaXzpqZdipnZiJQSEC8CL0fEYQBJkyW1RsQvy1pZDYgI2jJZLj1zDpLHH8ysupQyBvFDYKBgOZessxPY+tp+du0/4vEHM6tKpQREQ0T0Di4kr5vKV1LtaPP9D2ZWxUoJiC5J1wwuSLoW2FXKziVdJek5SVsl3Vzk/a9J2ph8PS+pp+C9D0p6Ifn6YCnfr9K0Z7IsmDmZRbOnpF2KmdmIlTIG8VHg+5L+MlnuBIreXV1IUj1wB3BFss16SfdHxObBNhHxyYL2nwCWJ69nA58DVpC/guqJZNvdJR1VBRgYCNo7slx5zpvSLsXMbFRKuVEuA6yU1AIoIkqdj/oiYGtEdABIuge4Fth8nParyYcC5Oe/figiupNtHwKuAu4u8XunbvPLe9lzqI9LzvT4g5lVpxN2MUn6U0kzI2J/ROyTNEvS/yxh3wuAHQXLncm6Yt/jDGAx8PBItpV0o6QNkjZ0dXWVUNL4GZz/YdUSjz+YWXUqZQzi6oh4fWwg6eb5zRK2K3ZdZxRZB3A9cF9E5EaybUTcGRErImLFvHnzSihp/LRlsiyZO5VTZjSnXYqZ2aiUEhD1kiYNLkiaDEwapv2gTmBRwfJCYOdx2l7P0d1HI9m24vTnBnh8WzerfHmrmVWxUgLiLuBnkv5A0h8ADwF/U8J264FlkhZLaiIfAvcPbSTpzcAsoL1g9YPAlUl31izgymRdVXjmpT3sP9LvgDCzqlbKIPWXJT0N/Ab5rp//C5xw5puI6Jd0E/kP9nrgOxGxSdLtwIaIGAyL1cA9EREF23ZL+iL5kAG4fXDAuhoM3v+wcokDwsyqVymXuQK8Qv5u6uuAbcCPStkoItYAa4asu23I8uePs+13gO+UWF9Fac9kecsp05jbUkpPnJlZZTpuQEg6i3y30GogC/wd+ctcLx+n2qrSkf4cG7Z3s/qi09MuxczspAx3BvEL4F+B34qIrQCSPjlMewM2vtjD4b4BVrl7ycyq3HCD1L9LvmvpEUnfkvTrFL/81Aq0ZbLUCS52QJhZlTtuQETETyLifcBbgH8GPgm8SdI3JV05TvVVnfZMlvMWzGDG5Ma0SzEzOyknvMw1Ig5ExPcj4j3k70fYCBzz4D2DQ705ntyx291LZlYTRjQndUR0R8RfRcS7ylVQNduwvZu+XPj+BzOrCSMKCBteWyZLQ524sHV22qWYmZ00B8QYas9keduimUydVOrtJWZmlcsBMUb2Hu7j6c4eTy9qZjXDATFG1m/rZiDw+IOZ1QwHxBhpz2RpaqjjV06flXYpZmZjwgExRtoyWd5++iyaG+vTLsXMbEw4IMbA7gO9bH55r8cfzKymOCDGwOD0op5/2sxqiQNiDLR3ZJnSVM/5C2emXYqZ2ZhxQIyBtkyWC1tn01jv/51mVjv8iXaSXtt7mK2v7ff4g5nVHAfESWpPxh98/4OZ1RoHxElqz2SZ1tzAuafNSLsUM7Mx5YA4SW2ZLCuXzKG+znMpmVltcUCchM7dB3mx+6DnfzCzmuSAOAntGd//YGa1ywFxEtozWeZMbeKs+dPSLsXMbMw5IEYpImjvyLJy6RzqPP5gZjXIATFKv8we5OU9hz3+YGY1q6wBIekqSc9J2irp5uO0uU7SZkmbJP2gYP2Xk3VbJP25pIr6M70tswvAN8iZWc0q29yYkuqBO4ArgE5gvaT7I2JzQZtlwC3ApRGxW9L8ZP0lwKXA+UnTfwN+FfjnctU7Um2ZLKdMb2bx3Klpl2JmVhblPIO4CNgaER0R0QvcA1w7pM2HgTsiYjdARLyWrA+gGWgCJgGNwKtlrHVEIoJ1mSyrls6hwk5szMzGTDkDYgGwo2C5M1lX6CzgLEn/LmmdpKsAIqIdeAR4Ofl6MCK2DP0Gkm6UtEHShq6urrIcRDHPv7qf7IFeP17DzGpaOQOi2J/WMWS5AVgG/BqwGvhrSTMlnQmcDSwkHyrvkvTOY3YWcWdErIiIFfPmzRvT4ofj8QczmwjKGRCdwKKC5YXAziJt/j4i+iJiG/Ac+cB4L7AuIvZHxH7gn4CVZax1RNozWRbNnszCWVPSLsXMrGzKGRDrgWWSFktqAq4H7h/S5qfA5QCS5pLvcuoAXgR+VVKDpEbyA9THdDGlITcQrOvIcsmSuWmXYmZWVmULiIjoB24CHiT/4X5vRGySdLuka5JmDwJZSZvJjzl8OiKywH1ABngGeAp4KiL+oVy1jsTmnXvZe7jfj9cws5pXtstcASJiDbBmyLrbCl4H8MfJV2GbHPCRctY2Wu0d+fEH3yBnZrXOd1KPUFsmy9J5U5k/vTntUszMysoBMQJ9uQEe39bNJUs9/mBmtc8BMQJPd/ZwsDfny1vNbEJwQIzA4PwPF3v8wcwmAAfECLRlspx96nRmT21KuxQzs7JzQJTocF+ODdt3u3vJzCYMB0SJnnyxh97+AV/eamYThgOiRO2ZXdQJLloyO+1SzMzGhQOiRG2ZLG9dOJPpzY1pl2JmNi4cECU42NvPxh09Hn8wswnFAVGC9b/cTf9AePzBzCYUB0QJ2jK7aKwXK1pnpV2Kmdm4cUCUoD2TZfmiWUxpKuuzDc3MKooD4gT2HOrj2Zf2sNLjD2Y2wTggTuDxbd0MhKcXNbOJxwFxAm2ZXUxqqGP56TPTLsXMbFw5IE6gPZNlRessJjXUp12Kmdm4ckAMI7v/CL94ZZ/nfzCzCckBMYx1Hd0ArPL4g5lNQA6IYbR37KJlUgPnL5iRdilmZuPOATGMtkyWC1tn0VDv/01mNvH4k+84Xt17mI6uAx5/MLMJywFxHIPTi3r8wcwmKgfEcbRldjFjciPnnDo97VLMzFJR1oCQdJWk5yRtlXTzcdpcJ2mzpE2SflCw/nRJayVtSd5vLWetQ7VlsqxcMpu6Oo3ntzUzqxhle/qcpHrgDuAKoBNYL+n+iNhc0GYZcAtwaUTsljS/YBffA74UEQ9JagEGylXrUDu6D9K5+xAfvmzJeH1LM7OKU84ziIuArRHRERG9wD3AtUPafBi4IyJ2A0TEawCSzgEaIuKhZP3+iDhYxlqP4vEHM7PyBsQCYEfBcmeyrtBZwFmS/l3SOklXFazvkfRjSU9K+kpyRjIu2jK7mNvSxLL5LeP1Lc3MKk45A6JY530MWW4AlgG/BqwG/lrSzGT9ZcCngAuBJcCHjvkG0o2SNkja0NXVNSZFRwRtmSyrls5F8viDmU1c5QyITmBRwfJCYGeRNn8fEX0RsQ14jnxgdAJPJt1T/cBPgV8Z+g0i4s6IWBERK+bNmzcmRXfsOsBr+4748d5mNuGVMyDWA8skLZbUBFwP3D+kzU+BywEkzSXftdSRbDtL0uCn/ruAzYyDtsHxB88/bWYTXNkCIvnL/ybgQWALcG9EbJJ0u6RrkmYPAllJm4FHgE9HRDYicuS7l34m6Rny3VXfKlethdozuzhtRjNnzJkyHt/OzKxilXWS5YhYA6wZsu62gtcB/HHyNXTbh4Dzy1nfUAMDwbqObi5/83yPP5jZhOc7qQs89+o+ug/0+vJWMzMcEEdp8/0PZmavc0AUaM/sonXOFBbMnJx2KWZmqXNAJPpzAzzW0e2zBzOzhAMisWnnXvYd6WeV538wMwMcEK/z/Q9mZkdzQCTaO7Kc9aYW5k2blHYpZmYVwQEB9PYPsH5bt88ezMwKOCCApzp7ONSX8/iDmVkBBwT5+R8kWLlkdtqlmJlVDAcE+fkfzjl1OjOnNKVdiplZxZjwAXG4L8fPt/f48d5mZkNM+IDYe7iPq847hcvfMv/Ejc3MJpCyPs21Gsyf1syfr16edhlmZhVnwp9BmJlZcQ4IMzMrygFhZmZFOSDMzKwoB4SZmRXlgDAzs6IcEGZmVpQDwszMilJEpF3DmJDUBWw/iV3MBXaNUTlpqpXjAB9LpaqVY6mV44CTO5YzImJesTdqJiBOlqQNEbEi7TpOVq0cB/hYKlWtHEutHAeU71jcxWRmZkU5IMzMrCgHxBvuTLuAMVIrxwE+lkpVK8dSK8cBZToWj0GYmVlRPoMwM7OiHBBmZlaUAyIh6YuSnpa0UdJaSaelXdNoSfqKpF8kx/MTSTPTrmm0JP0nSZskDUiquksSJV0l6TlJWyXdnHY9J0PSdyS9JunZtGs5GZIWSXpE0pbkZ+uP0q5ptCQ1S3pc0lPJsXxhTPfvMYg8SdMjYm/y+g+BcyLioymXNSqSrgQejoh+Sf8LICI+k3JZoyLpbGAA+CvgUxGxIeWSSiapHngeuALoBNYDqyNic6qFjZKkdwL7ge9FxHlp1zNakk4FTo2In0uaBjwB/HY1/rtIEjA1IvZLagT+DfijiFg3Fvv3GURiMBwSU4GqTc6IWBsR/cniOmBhmvWcjIjYEhHPpV3HKF0EbI2IjojoBe4Brk25plGLiEeB7rTrOFkR8XJE/Dx5vQ/YAixIt6rRibz9yWJj8jVmn10OiAKSviRpB/AB4La06xkjvw/8U9pFTFALgB0Fy51U6QdRrZLUCiwHHku3ktGTVC9pI/Aa8FBEjNmxTKiAkPT/JD1b5OtagIj4k4hYBHwfuCndaod3omNJ2vwJ0E/+eCpWKcdSpVRkXdWemdYaSS3Aj4D/NqQHoapERC4iLiDfU3CRpDHr/msYqx1Vg4j4jRKb/gB4APhcGcs5KSc6FkkfBN4D/HpU+EDTCP5dqk0nsKhgeSGwM6VarEDSX/8j4PsR8eO06xkLEdEj6Z+Bq4AxuZBgQp1BDEfSsoLFa4BfpFXLyZJ0FfAZ4JqIOJh2PRPYemCZpMWSmoDrgftTrmnCSwZ2vw1siYivpl3PyZA0b/AqRUmTgd9gDD+7fBVTQtKPgDeTv2JmO/DRiHgp3apGR9JWYBKQTVatq+Irst4L/AUwD+gBNkbEu9OtqnSSfhP4OlAPfCcivpRySaMm6W7g18g/WvpV4HMR8e1UixoFSe8A/hV4hvzvO8D/iIg16VU1OpLOB/6G/DFurWMAAAIgSURBVM9XHXBvRNw+Zvt3QJiZWTHuYjIzs6IcEGZmVpQDwszMinJAmJlZUQ4IMzMrygFhNgKS9p+41bDb3ydpSfK6RdJfScokT+J8VNLFkpqS1xPqRlarPA4Is3Ei6VygPiI6klV/Tf7hd8si4lzgQ8Dc5MF+PwPel0qhZgkHhNkoKO8ryTOjnpH0vmR9naRvJGcE/yhpjaT/mGz2AeDvk3ZLgYuBz0bEAEDy1NcHkrY/TdqbpcansGaj8zvABcDbyN9ZvF7So8ClQCvwVmA++UdJfyfZ5lLg7uT1ueTvCs8dZ//PAheWpXKzEvkMwmx03gHcnTxJ81XgX8h/oL8D+GFEDETEK8AjBducCnSVsvMkOHqTCW3MUuGAMBudYo/yHm49wCGgOXm9CXibpOF+BycBh0dRm9mYcECYjc6jwPuSyVrmAe8EHic/5ePvJmMRbyL/cLtBW4AzASIiA2wAvpA8XRRJywbnwJA0B+iKiL7xOiCzoRwQZqPzE+Bp4CngYeC/J11KPyI/D8Sz5OfRfgzYk2zzAEcHxn8BTgG2SnoG+BZvzBdxOVB1Txe12uKnuZqNMUktySTyc8ifVVwaEa8kz+t/JFk+3uD04D5+DNxSxfNxWw3wVUxmY+8fk0lcmoAvJmcWRMQhSZ8jPy/1i8fbOJlc6KcOB0ubzyDMzKwoj0GYmVlRDggzMyvKAWFmZkU5IMzMrCgHhJmZFfX/AbBtHqr/vk5CAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid.cv_results_[ 'mean_test_score' ]\n",
    "\n",
    "print(test_means)\n",
    "\n",
    "plt.plot(np.log10(Cs), test_means,label='test')\n",
    "\n",
    "plt.legend()\n",
    "plt.xlabel( 'log(C)' )                                                                                                      \n",
    "plt.ylabel( 'Accuracy' )\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "随着C的增大，模型越来越复杂，交叉验证得到的测试集正确率先增加后减小，因为当C增大到一定程度时，模型出现过拟合现象，所以测试集的正确率就减小了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "大家可以对比一下用原始数据（标准化，但不做缺失值填补，即缺失值取0），效果更好。\n",
    "SVM对噪声比较敏感（或许原始数据清洗的时候已经是上帝最好的安排了:) ）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RBF核SVM正则参数调优\n",
    "\n",
    "RBF核是SVM最常用的核函数。\n",
    "RBF核SVM 的需要调整正则超参数包括C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和核函数的宽度gamma\n",
    "C越小，决策边界越平滑； \n",
    "gamma越小，决策边界越平滑。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score=nan,\n",
       "             estimator=SVC(C=1.0, break_ties=False, cache_size=200,\n",
       "                           class_weight=None, coef0=0.0,\n",
       "                           decision_function_shape='ovr', degree=3,\n",
       "                           gamma='scale', kernel='rbf', max_iter=-1,\n",
       "                           probability=False, random_state=None, shrinking=True,\n",
       "                           tol=0.001, verbose=False),\n",
       "             iid='deprecated', n_jobs=4,\n",
       "             param_grid={'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000],\n",
       "                         'gamma': [0.0001, 0.001, 0.01, 0.1, 1]},\n",
       "             pre_dispatch='2*n_jobs', refit=True, return_train_score=False,\n",
       "             scoring=None, verbose=0)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "gammas = [0.0001,0.001, 0.01, 0.1, 1]\n",
    "#gammas =[1e-5, 1e-6]\n",
    "param_grid = {'C': Cs, 'gamma' : gammas}\n",
    "grid = GridSearchCV(SVC(kernel='rbf'), param_grid, cv=5, n_jobs = 4)\n",
    "\n",
    "grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7682539682539682\n",
      "{'C': 100, 'gamma': 0.001}\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOydd3xUVdrHv2cmkzrpvZKQhNAJvajYUBQVXQuCiiIoupZ13dVXd91VV13ddX3Vtb2uih2l2Bv2spqQQIDQCQktvZCeSZlk5rx/3BASDKTN5M4N9/v5zCczd84985tkMs895zzP7wgpJTo6Ojo6OsdiUFuAjo6Ojo5rogcIHR0dHZ1u0QOEjo6Ojk636AFCR0dHR6db9ACho6Ojo9MtbmoLcBQhISEyPj5ebRk6Ojo6mmLTpk2HpZSh3T03ZAJEfHw8WVlZasvQ0dHR0RRCiEPHe06fYtLR0dHR6RY9QOjo6OjodIseIHR0dHR0umXIrEF0R2trK4WFhTQ3N6stxaXw9PQkJiYGk8mkthQdHR0XZkgHiMLCQnx9fYmPj0cIobYcl0BKSWVlJYWFhSQkJKgtR0dHx4UZ0lNMzc3NBAcH68GhE0IIgoOD9VGVjo5OjwzpAAHowaEb9N+Jjo5ObxjSU0w6Ojo6HbQ2Q20BVB+CmoPQZoXRF4N/tNrK+kVlQws5pfXklNXj4WbkqulxDn8NPUAMAl9++SV33HEHNpuNG264gXvvvbfL8y0tLVx77bVs2rSJ4OBgVq9ezZGq8Mcee4wVK1ZgNBp55plnmDt37gn7fO6553j66afZt28fFRUVhISEDOp71dFRDbsd6kug+iDUHFICQef79cW/Pufr+yDpHJh0LYyYC0bXS9ywtLSxt6yevWX17ClVfuaU1nO4wdrRZlJcgB4gtIjNZuPWW2/lm2++ISYmhqlTpzJ//nxGjx7d0WbFihUEBgaSl5fHqlWruOeee1i9ejW7du1i1apV7Ny5k+LiYubMmcPevXsBjtvnKaecwoUXXsgZZ5yh0jvWOZb9tfupt9aTHJCMt8lbbTnaRUpoqm7/wj/YPhLodL+2AGzWTicI8IuGwGEw/AwIjFfuBwxTfrY1w5aVkL0SVl8NPmGQepUSLIITB/3tWdvsHDhsYU9pXUcQyCmrp6CqqaONl8nIiHAzZ6aEkRLhq9zCfQn19XCKJj1AOJkNGzaQlJTE8OHDAVi4cCEff/xxlwDx8ccf8+CDDwJw+eWXc9tttyGl5OOPP2bhwoV4eHiQkJBAUlISGzZsADhunxMnThzcN6hzQmx2G9etu46alhoAos3RJAckkxyo3JICkoj3j8dkcL0rV1VobYKa/GOu/g8eHQW01HVt7xWofPFHjINRFyr3A4YpP/1jwK2HL86z/wpn/AnyvoXNb0L6s5D2NAw7VQkUo+eDycuhb9FulxRWN5FTVk9OaR05ZQ3klNaxv8JCm13Z4dNoEAwP8WF8TAALJscyIsKXkRG+xAZ6YzAM3hriSRMg/vbpTnYV1/XcsA+MjvLjgYvGnLBNUVERsbGxHY9jYmLIzMw8bhs3Nzf8/f2prKykqKiIGTNmdDm3qKgIoMc+dVyD3VW7qWmpYcmYJZhNZvJq8sitzuXnop+xSRsAbgY3EvwTSApIYkTgCJICkkgOTCbKJ2roJRTYbVBX1Onq/5gA0FDatb2b59Ev/LiZR6/+jwQCT7+BazK6Qcp5yq2+FLLfUYLFh8vhi7th/AIlWESO71O3UkoON1g7RgJHgkFuWT2NVltHu+gAL0ZG+HL2qHBGRvgyItyX4aE+eLgZB/7eBshJEyDUors9v4/9pz9em+Mdt9vtPfap4xpklGQAcN2Y6wjxOroeZLVZOVB7gNyaXHKrc8mrySO7PJt1B9Z1tPEx+ZAYkHh0xNH+M9AzcNDfR6+REhqr2r/0D3YzDVQI9taj7YUB/GKUL/2kOcdMA8WDOQwG87PtGwGn/QFOvRMO/gJb3lJuG1+GyFQlUIy7HDz9u5xW39zK3rKGo1ND7UGhynJ0yivIx52UcF8WTIntmB5KDjPj6+m6o8eTJkD0dKXvLGJiYigoKOh4XFhYSFRUVLdtYmJiaGtro7a2lqCgoBOe21OfOq5BRkkGSQFJXYIDgLvRnZSgFFKCUrocr7fWs69mH3ur93YEjm/zv+X93Pc72oR4hXSMMo4EjcSARLzcHDsVclyslm6mgTrdtzZ0be8drHzZR02EMZd0mgYaBv6xLrkwjBCQcJpyO/+fsP092PQGfP4H7F/9mcLI8/iv7/l8bxlOTlkDRTVH1wm83Y2MCPflnFHhHYFghBPXCZzJSRMg1GLq1Knk5uZy4MABoqOjWbVqFe+8806XNvPnz+eNN95g5syZvPfee5x11lkIIZg/fz5XXXUVf/jDHyguLiY3N5dp06YhpeyxTx31abG1kF2ezRUjruj1Ob7uvqSGpZIaltpxTErJ4abD5Fbndow4cmtyWZOzhhZbCwACQYxvTEfASApMYkTACOL84nAzOOjfvL4M3pwPFXu6Hjd5H73iTzjt6P3AYRAQBx6+jnn9QcRul+RXNbZPDdWTUzaZvY3J+LRu54q2H5if/xXXiI843RBNVvBFVE+8jNjYYYyM8CU6wGtQ1wmciR4gnIybmxvPPfccc+fOxWazsXTpUsaMGcP999/PlClTmD9/PsuWLWPx4sUkJSURFBTEqlWrABgzZgwLFixg9OjRuLm58fzzz2M0KvOS3fUJ8Mwzz/D4449TWlrK+PHjmTdvHq+88opq7/9kJrs8mxZbCzMiZ/Tc+AQIIQj1DiXUO5RZ0bM6jtvsNgobCpWA0Sl4/Fj4I3apTEOaDCaG+w/vsig+InAE4d7hfZ+W3PKWEhzO+BMEJx0NBD4hgzsN5ECklFTUt3RJH80pqye3rIGm1qPrBLFBXqSE+5Ey9hx8Iy6jJMiIV/k3xG59m9j8F6HyFaiZB6brIOBMQP31A0cgupvn1iJTpkyRx24YtHv3bkaNGqWSItdG/904n2c2P8OrO17ll4W/YHY3D9rrNrc1d6xv5FXnsbdGma4qbyzvaONr8iUpMOnoiKN9ysrfw7/7TqWEZyeBbyRc/8UgvRPnkVtWz/0f72RPaR3VjUfXRELMHqREmJVgEGEmJcKP5DAzPh4nuJau2Atb3oTsd6HxsLKmMvEamHi1MoJyEEdGkkUNRRQ2FFJUX9RxP8I7gkdPe7Rf/QohNkkpp3T3nD6C0NFxEpklmYwNGTuowQHA082TUcGjGBXc9QKgtqW2I4vqyM91B9axZu+ajjZhXmFdRhvJgckM9x+OZ9EWqNoPp901qO/FWfzv13vZUVTLhRMiSQn3ZUR7PUGwuR/rBKEj4NxH4Kz7Ye86JQPqp38qt8SzlIXtlHng5t5jV/XWeooaiiiqV774C+sLlccNRRQ3FNNs6+qhFuIVQow5hmCv4L7r7gV6gNBxSZp376bhl1/wSEzCY0QypqgohEE71mF11jp2VO7ghnE3qC2lA38PfyaHT2Zy+OSOY1JKyhrLugSN3JpcNu7eiNWuZOAYhIE44UFyeDhJtgpC964lyCOIIK8gAj0CCfIKwtfkq5lMusqGFr7dXcb1p8Rz3wWjez6ht7i5K9Ydoy+GmgKlAG/zW7D2OmWhfsIirBMWUexl7vjSL6wvVEYD7Y9rW2q7dGk2mYnxjSHBP4FTo08l2hxNjG8MMeYYosxReLp5Ok5/d2/JmZ0LIc4D/o0yIfeKlPIfxzz/FHBm+0NvIExKGdD+XBzwChALSGCelPKgM/XquA5ljz9O4/qMjsfC2xuPpCQ8kpPwSE7uuLmFhrrkF1NWaRZ2aR/w+oOzEUIQ4RNBhE8Ep8Wc1nG8zd5Gfn2+Ejgqd5O78f/IMfvz7c5Xkfx6WtrN4EaQRxCBnoEEeR79eeTW+XGgZyBmk1m1v9uHW4pos0sWTIntuXEfsUs75Y3lFLWUUxgziiL/31JUsonCyt0UFn5IRcnHyE7v22QwEW2OJtoczdjgsUT7RhNjjun46efup+rn22kBQghhBJ4HzgEKgY1CiE+klLuOtJFS3tmp/e1A5zLgN4G/Sym/EUKYgV8n/+sMSexNTTRt2kzAwisJuOQSmnNzacnNpWVvLg0//kTt+x90tDX6+yvBYkR70BgxAo+kJIz+x5lLHyQySzLxNHoyIXSCqjr6i5vBjeH+wxnuP5y5NZVQWgrXv4Y1ZgpVzVVUN1dT1VzVcTvy+MjPgvoCqpqraGxr7LZ/k8FEoGcgwZ7B3QaVIyOTIyMVbzdvh3xRSilZvbGAiXEBJIf3PbtKSkmdta7rlX+ntYDihmJaO9V5CARh3mFEh6QwwzOEmIZKoou2EV1dSLTwIGz0bzBMvg6iJrnkQr8zRxDTgDwp5X4AIcQq4GJg13HaLwIeaG87GnCTUn4DIKVsOM45OkOQxqxNSKsV37PPxis1Fa/U1C7Pt1VW0pKbpwSN9lvtJ59ibzj6MXELD+8y0vBITsYjKRGD1+DUCmSWZDIpfBLuxp7nnV2eLSshaDjEzcS904ijNzS3NStBo6VrEKlsruzy+FDdIaqaq2hqa+q2H3eDe5cprc6jle5GKV5uXt0GlOyCGnLLG3js0nHH1dzU1kRxQ3HHFFDnn0UNRTS0dv068vfwJ9oczYjAEZwVd5YyAmgfFUSZo379GZASCjKVtYpta2DzGxA+tr0I7wrwDurV73YwcGaAiAYKOj0uBKZ311AIMQxIAL5vPzQCqBFCfNB+/FvgXiml7ZjzlgPLAeLiHO9kqKMOlvR0hMmE95RuEytwCw7GLTgYnxlHP05SStpKS48Gjb25NOfupXHlBqS1vZpVCEyxse0BIwnPESPwSE7GPT4e4cDtVysaK9hXu4/5SfMd1qdqVB2AQ7/AWX/p1xWup5snkeZIIs2RvWrf1NZ03NFJ5/sHag5Q1Vz1q0Xbjtc1enY7MsnIbcY7yE5giBcZJcWUNJT8ajH4cNPhLn15GD06vvAnhU9S1gHap4GizdH4uvdxJCIExM1Qbuc9BjveV4LFuv+Br/+q+D9Nulbxg1J53c2ZAaK7T9PxcmoXAu91CgBuwGkoU075wGpgCbCiS2dSvgS8BEqa68AlOwdn2H0vXbqUzz77jLCwMHbs2DHYb8mpWNLS8Jo8uU9X+0IITJGRmCIjMc+e3XFc2mxY8/M7jTaUkUfDjz+Crf3jZjLhER/fdaoqORlTTEy/FsaP2GtMj+z2ekhbbH0XEDBh0aC8nJebF15mL6LMvXMGaGxtPOGUV1VLFZVNleTV5FHVVIXVbsUYDnf/fDRzyyAMRHhHEO0bzWnRpynBoH0NIMY3hmBPJ+5K6ekPU5Yqt5JtSq3JttWwfS0EJsCkxZB6tWIBogLODBCFKAvMR4gBujFkB5QAcesx527pND31ETCDYwKEFnCG3bfRaGTJkiXcdtttXHvttSq+O8fTWl5Oy969hP7xDw7pTxiNeCQk4JGQAOee23HcbrViPXCAlr1Hp6matm2j7oujOf7CywuPxMSu01QjknELCzvhF0ZmSSZ+7n6MDBzpkPegGna7kts//AzFGdUF8TZ5423yJsa3Z31rswq4+4ONPH/NCGJCbbS0tRBpjiTCJ8I13HQjx0Pkv+Cch2D3p8qo4ruH4Pu/K3tVTLpW2bvCOHjJp858pY1AshAiAShCCQJXHdtICJECBALrjzk3UAgRKqWsAM4Cso49Vws4w+575syZzJ49m4MHD6rwjpxL43rlY+Aza1YPLQeGwd0dz5QUPFO6eiHZGixY93Vd32j45WdqP/zw6Ln+/l2yqTzbfxoDApBSklmayfTI6RgNGq+mPfhfqM2HOQ+orcQhrN1USEJQEPNGjXHJzLcOTF6Kg+z4BVC5r90wcCXkfAHmCKUAb+I1yrqQk3FagJBStgkhbgO+QklzfVVKuVMI8RCQJaX8pL3pImCV7FTSLaW0CSHuAr4Tyl9yE/DygAStuxdKtw+oi18RMQ7O/8cJmzjL7nuo0pCWhjEwEE+VqryNZh+8JkzAa0LX7KO26uouQaMlN4+6zz7HXl/f0cYtNBT78FjOtRcxYfokmoK24ZGYiMHHZ7DfhmPYshI8/GHkBWorGTAHDlvYcKCK/zkvxbWDw7EEJ8KcB+HM+yD3a2VU8ctT8PP/QsJsmHQdjLwQTM6ph3DqWEVK+QXwxTHH7j/m8YPHOfcboG8G7C6IM+y+hypSSizp6/GZNcvliuLcAgNxmzYNn2nTOo5JKWkrK+tYFG/JzaVk+wbOOSTx2PgJB59TroFMsbGYzziD4OuXYNKK625zLez+RNlhzcEb5qjB2qwCDAIum+SaU2U9YjQpgXrkBVBXfLQI7/1l4BmgrFWc87DDU2VPnkrqHq70nYWz7L6HIi1792I7fNjp00uOQgiBKSICU0QE5tOUIrMnfvwD28vgs+krsOYpU1VNO3dS/e67VL/7Lv7z5xN8ww14DE9QWX0P7PhA2ZIz9Rq1lQyYNpud9zYVcmZKGOF+zq08HhT8omD23XDqH5VpwM1vQkOFU+ooTp4AoRLOsPseqljS0gHwOUUbAeJY7NLOhtINnB5zupIVFR+P75w5ALQWFVH52uvUrF1L7Ycf4nvuuQQvvxGvMersU9Ij2SshdCRET1JbyYD5b24F5fUtXOGEymlVMRiUBILhZyi1Fc54Caf0qtNBZ7vvUaNGsWDBgg67708+UaYgli1bRmVlJUlJSTz55JP84x/KaKez3fd5553Xxe570aJFzJw5k5ycHGJiYlixQnMJXr/CkpaGe2Iipgh1UvoGyp6qPdS21HZrr2GKjibiL/eR9P13BC9fjiUtjYOXXU7+shuwbNjQ7XSialTkQOFGJb1yCExprt5YQIjZnbNHhaktxXk46e+k232fpLja78be0sLeadMJuHIBEX/+s9py+sVrO17jyU1P8t0V3xHmfeIvI1t9PdXvrqLq9dexVVXhNXEiwctvxHzGGeqvM31zP6Q/B3/YDb7h6moZIIcbWpjx6HeON+YbQpzI7lsfQei4BE2bNiFbWjSz/tAdmSWZDPcf3mNwADD6+hKy/EaSvv+O8L/+hdayUgp/ewsHLvkNtZ99jmxrGwTF3WBrg62rIflczQcHgA83O8+Y72RADxA6LoElPR1MJnymTlVbSr9otbWyuXxzn6unDZ6eBF19NUlffUXkPx5DtrZSfNdd7Jt3AdWr12C3WnvuxJHs+x4aSpVce40jpWRNVv+N+XT0AKHjIjSkpeOdmqrZmoGtFVtpamvqt72GMJkIuOQShn/2KdHPPoPRz4/SBx5g35xzqHz1NewWi4MVH4fst5W9C5LnDs7rOZEt7cZ8V+qjh36jBwgd1Wk7fJiW3bvxOeUUtaX0m4ySDAzCwNSIgY2AhMGA3znnEL92DXGvrsA9IYHyxx8n76yzqXj2Odqqqx2kuBsaqyBnHYxb0Kvdz1ydtVkFeJmMXDC+d0aBOr9GDxA6qmNp3xhIywEisySTMcFj8HP3c0h/Qgh8Zs1i2BuvE7/qXbwmT+bw88+Td/Ycyv75OK1l5T130le2rwWbdUhMLzVa2/h0awkXjI/E19MFfJY0ih4gdFTHkpaG0d8fz9Guk1XVFyytFnYc3uE091av1FRiX3iehE8+xvfss6l68032zZlDyf0PYM3Pd9wLbXkbIsYrFjIa54vtpTS0tOmL0wNEDxCDwJdffklKSgpJSUkdNQ6daWlp4corryQpKYnp06d3mPBVVlZy5plnYjabue222wZZ9eCg2Guk4z1rJsKoTXO7TWWbaJNtTrf39hwxguh/PU7il+vwv+xSaj/6iH3nnU/RH++iOSdnYJ2XbofSbYoJ3BBgzcYCEkJ8mBofqLYUTaMHCCdzxO573bp17Nq1i3fffZddu7puqtfZ7vvOO+/knnvuAcDT05OHH36YJ554Qg3pg4I1L4+28nJNp7dmlGTgbnAnNTS158YOwD02lsgHHyTx228Iun4JDT/8wIGLL6Hg5t/SuHlL/zrdshKM7sqOZhpnf0UDGw5WccWUGPVrSjSOHiCcTGe7b3d39w677858/PHHXHfddYBi9/3dd98hpcTHx4dTTz0VT88h4B9zHCzpir2GWeMBYmLYRDzdBvfvZAoLI/zuu0n64XtCfnc7TdnZHLrqKg4tvpaGn3/pfXV2mxW2r4GU811qu8v+snZTIUaD4HKtGvO5ECeNF9M/N/yTPVV7HNrnyKCR3DPtnhO2GYjdd0hIiEP1uiINaWm4x8djio5WW0q/qGyqJLc6lzsm3aGaBqO/P6G33ELwkiXUrF1L5auvUXDjjXiOHk3w8uX4njPnxNN3uV9BY+WQMeZ7f1MhZ4wIJWwoGPOpjD6CcDIDsfse6titVho3Zmk6e2lD6QYApkeov72owduboOuuI/Gbr4l85GHsFgtFv/89+y+8iJr3Pzi6N/exbFmpbESTeNbgCnYCP+1VjPkWTNUXpx3BSTOC6OlK31kMxO57qNO0eQuyqUmz7q2gpLf6mnwZHew6Pj8Gd3cCLr8c/9/8hvqvv+bwSy9Tct99VDz3HMFLlxJw+WVH9/uuL1M2opl1+6BuZeks1mQpxnxnjRzCxnyDiD6CcDKd7b6tViurVq1i/vz5XdocsfsGuth9D3UsaWng5oa3hi3MM0oymBIxxSW3FxVGI37nn0/CB+8T+9J/MEVHUfb3v5N39hwOv/gfbHV1sG01SJvi3KpxKupb+G53OZdOisFk1L/aHIH2LxlcnM523zabjaVLl3bYfU+ZMoX58+ezbNkyFi9eTFJSEkFBQaxatarj/Pj4eOrq6rBarXz00Ud8/fXXXfaz1jKW9HS8UidgNJvVltIvCuoLKGooYvHoxWpLOSFCCMyzZ2OePZvGTZs4/NJLVDz9NJWvvEJgipWgaZNwCx2htswB89GWI8Z8+uK0o9ADxCAwb9485s2b1+XYQw891HHf09OTtWvXdnvukZqIoUZbdTXNu3YRcrt26zsyS5Rkg+72f3BVvCdPJu4//6F5924q//04lT+up2rbYQJqHyZo6VLcY7SZLCClZHVWAZPiAkgK0435HIU+DtNRhcb160FKzBpeoM4sySTUK5Th/sPVltJnPEeNIvrCABLn1+F/4Tyq165l39y5FN9zLy15eWrL6zNbCmrIK2/gSn1x2qHoAUJHFRrS0jD4+eE5dqzaUvrFke1Fp0dO1+Z6UWsTbH8f9+kXEPnYP0n65muCrrmGuq+/Zv+FF1F4++00bd+utspes2ZjAd7uRi4YP3T3bFcDPUDoDDqKvcZ6fGbM0Ky9Rm51LlXNVU6313Aaez6HltoOYz5TRAThf7qXpO+/I+SWW7BkbuDgFQvIX7oUS0ama22JegyKMV8xF4yLxOyhz5o7Ej1A6Aw61gMHaCsp0bS9hhbXH7qQvRL84yB+dpfDboGBhP7udpK+/56wu++mOTeX/CVLOLhwIfXffYe021USfHw+31aCxWrTax+cgB4gdAYdyy9pAPicquH1h9JMhvkNI8InQm0pfae2EPb9AKmLwND9V4DR7EPwsqUkffstEQ8+iK2yisJbb+PAxZfQvHv3IAs+MWuzChke4sOUYboxn6PRA4TOoGNJT8cUF4d7jDbTEVvtrWSVZml39LD1XUDChEU9NjV4eBC48EoSv1xH1L/+ha2+nkPXLaEpO9v5OnvBUWO+WG2uBbk4Tg0QQojzhBA5Qog8IcS93Tz/lBAiu/22VwhRc8zzfkKIIiHEc87U6Wx6svv+73//y6RJk3Bzc+O9995TQeHgIa1WLBs2aLp6esfhHTS2NWpz/UFKyH4Hhp0KQQm9Pk24ueF/0YXEr3wbY2AAh5Yuw5K5wYlCe8eaLMWY77JJ2kzPdXWcFiCEEEbgeeB8YDSwSAjRpcJLSnmnlDJVSpkKPAt8cEw3DwM/OUvjYNAbu++4uDhef/11rrrqKpVUDh5NW7ciGxs1nd6aUZKBQDAtQoMV4PnroWp/v3eNM0VHM+ytt3CPjqJg+XIaflLv37PNZuf9zYWcmaIb8zkLZ44gpgF5Usr9UkorsAq4+ATtFwHvHnkghJgMhANfO1Gj0+mN3Xd8fDzjx4/HcJz54KFEQ1oaGI14T9fg1Xc7mSWZjAwaib+Hv9pS+s6WleBuhtEn+lc8MaawMOLefBOPxEQKbruduq/U+Rf9MaeCivoWfdc4J+LMnLBooKDT40Kg228FIcQwIAH4vv2xAfhfYDFw9vFeQAixHFgOylX4iSh99FFadjvW7ttj1Egi/vznE7bpjd33yYQlfT1e48dj9NVmtWtjayNbK7ayeJRr22t0S0sD7PwQxv4G3H0G1JVbYCBxb7xOwfKbKLrzTuRjj+J/cf+DTn9QjPk8OFM35nMazrxk7W7F6HjJ1AuB96SUtvbHtwBfSCkLjtNe6UzKl6SUU6SUU0JDQwcg1XmcrFbe3WGrqaF5+3ZNp7duLt9Mm93524s6hV0fQ6vFYfs+GH19iVvxCt7Tp1F8z71Ud/IQczYV9S18v6ecyyZF68Z8TsSZI4hCoPPYLwYoPk7bhcCtnR7PBE4TQtwCmAF3IUSDlPJXC929pacrfWfRG7vvkwVLRiZIqen9HzJLMjEZTEwMm6i2lL6TvRKCEiHOcdlXBm9vYl98kaI7fk/pg3/D3thE8NLrHdb/8fhwSyFtdskV+vSSU3Fm6N0IJAshEoQQ7ihB4JNjGwkhUoBAYP2RY1LKq6WUcVLKeOAu4M2BBAc16Y3d98mCJS0Ng9mM1/hxakvpN5klmUwInYC3yVttKX2jaj8cSoPUq8DBI1iDhwcxzz6D7/nnUf7441Q897xTK6+llKzeWMDkYYEkhWnTCVgrOC1ASCnbgNuAr4DdwBop5U4hxENCiM7fkIuAVdKVa/kHQGe771GjRrFgwYIOu+9PPlHi5caNG4mJiWHt2rXcdNNNjBkzRmXVjkdKiSUtDe8Z0xFu2rRDqGmuYU/VHm1OL2W/C8LQq9qH/iBMJqKfeAL/Sy/l8HPPUf6vJ5wWJDbn17CvwqLbeg8CTv1PlVJ+AXxxzLH7j3n8YA99vA687mBpg0pPdt9Tp06lsLBwsKGYHYYAACAASURBVGUNKq2HDtFaXEzQDcvUltJvNpRuQCK1VyBntyvFccPPBH/n1QsIo5HIRx7G4OVF1auvYm+0EHH//QgHZ+fpxnyDhzYv5XQ0R0OaYq+h5fqHzJJMvN28GROisRHegZ+gtgDO+ZvTX0oYDIT/5T4M3t5UvvwysqmJyL//3WGjRktLG59t0435Bgv9N6wzKFjS12OKicHUQzqyK5NRksHUiKmYDCa1pfSN7JXg6Q8pFwzKywkhCPvjHzD4eFPx9L+xNzUT/cS/EO7uA+778+2KMZ++78PgoOeH6Tgd2dpKY0YGPrNmaTbFt6ShhPz6fO2tPzTVwO5PYezlYBrcauOQm28m/E/3Uv/11xTcdhv25uYB97k2q4DhoT5M1o35BgU9QOg4nabt27FbLJpOb80oyQDQXoDY+QG0NffbWmOgBF13HREPP4Tl518oWH4TtgZLv/vaV9HAxoPVLNCN+QYNPUDoOB3LL2lgMOAzQ2Nfrp3ILM0kyDOI5IBktaX0jS0rIXQURE1STULgFVcQ9fjjNG7aRMGyZdhqa/vVz9p2Y75LdWO+QUMPEDpOx5Kejue4sRj9NehdhJKim1mSyfQIjW0vWpEDRVnK6EFl3f4XXUjMv5+medcuDi25nraqqj6df9SYL4wwX92Yb7DQA8QgsHTpUsLCwhir0f2XB4Ktro6mbds0ba+xr2Yfh5sOa296KXslCCOMv1JtJQD4zplDzAsvYD1wgEPXLKa1rKzX5x415tNrHwYTPUAMAkuWLOHLL79UW4YqWDIywG7Xdnprafv2olEaqn+wtcHWVTBiLphdx8zOfNqpxL38Em1lZRy6ZjHWwqJenbdaN+ZTBT1ADAKzZ88mKChIbRmqYElPx+DtjdeECWpL6TcZJRnEmGOINmto7nvfd9BQBqnqLE6fCO+pU4l7/TVsdXUcuvpqWvYfOGH78vpm3ZhPJU6aOoif1+zlcEGDQ/sMiTVz2oIRDu1zqGFJS8d7+nSESWO1A+202dvIKs1ibvxctaX0jS1vg3eIMoJwQbzGjWPYm2+Qv3QZhxYvJu7VFXimpHTb9sPNRdh0Yz5V0MOxjtOw5ufTWlCg6fTWXZW7aGht0Ja9hqUSctYpaw9G1w3MnikpDHvrLYTJxKFrr6Np27ZftZFSsiZLN+ZTi5NmBKFf6Q8+lvR0AE0vUGeWKOsP0yI1tL3o9rVgb1WcW10cj+EJDHv7bfKvv57865cS++L/4T11asfzm/Or2Vdh4fHLElVUefKijyB0nIYlLR23qEjcE+LVltJvMkoySAlMIchTQ2tI2W9D5ASI0EbWnHtMNMPefgu38HDyb1xOw8+/dDy3ZmMh3u5G5o2PVFHhyYseIAaBRYsWMXPmTHJycoiJiWHFihVqS3I6sq0Ni8btNZrbmskuz9ZWemvJNijd7rBd4wYLU3g4w95+C/eEBApvuYX6b7/tMOa7cLxuzKcW+m99EHj33XfVljDoNG3fjr2+XtPprVvKt2C1W7UVILJXgtEdxl2utpI+4xYUxLA3XqfgxuUU3vF78m+6G4s1hAX64rRq6CMIHadgSU8HIfCeoaHF3WPILMnETbgxJXyK2lJ6R5sVtq2BlHngraEpsU4Y/fyIXbEC78mTiXnhn1xTuUU35lMRPUDoOAVLWjqeY8bgFqjdf+7MkkzGhY7Tzvaie7+EpiqYqK3ppWMxmn1offR/yQpP4eqfV1L95ptqSzppGfIBYojuZDognP07sTU00LR1q6azl2pbatlVtUt700u+kZB4ltpKBsza7RU8OvN63M86m7LH/sHhF1/U/5dVYEgHCE9PTyorK/UPVieklFRWVuLp6TzDs8bMTLDZNF3/kFWahV3atVP/UF8Gud/AhIVgMKqtZkC02uy8v6mI2aOjGP7M0/hffDEVT/+biief1P+XB5khvUgdExNDYWEhFRUVaktxKTw9PYmJcZ7pmSUtHeHtjdfEVKe9hrPJKMnAy82L8SHj1ZbSO7atAmnTXPZSd/yYU8HhhhZl3wc3NyIfexTh5Unly69gb2wi/L4/O3yfa53uGdIBwmQykZCQoLaMkw5LWhreU6dgcMAWk2qRWZrJpPBJmFy4ErkDKZV9H2KnQ0iS2moGzJqsAkJ9PTgzJRRQ9rmOeOABDN4+VL36KvbGRiIfeRhh1PZISQvoYVjHoVgLi7AeOqTp9NYySxkHag8wI0Ij00tFm+BwjiYqp3viiDHfpZOicetkzCeEIOzuuwi57TZqP/yQorvuQlqtKio9ORjSIwidwceSngZo215jQ+kGQEPbi255G9y8YMylaisZMEeM+bqrfRBCEHrbrRi8vSl//HEKm5qJ/vfTGDw8VFB6cqCPIHQciiUtHbfwcNwTteudk1GSQYBHAClB3buLuhStTbDjAxg9Hzz91FYzIKSUrM4qYMqwQBJDj2/MF7z0eiIefICGn36i4OabsVv6v8+1zolxaoAQQpwnhMgRQuQJIe7t5vmnhBDZ7be9Qoia9uOpQoj1QoidQohtQgjX2BJL54RIm03z9hpSSjJKMpgWMQ2D0MD1057PoaXWJfd96CubDlWzv8LCgqk9V04HLlxI1D8eozFzA/k33Iitvn4QFJ58OO0/QAhhBJ4HzgdGA4uEEKM7t5FS3imlTJVSpgLPAh+0P9UIXCulHAOcBzwthAhwllYdx9C8cyf22lpNp7cerDtIeWO5tqaXAuIg/jS1lQyYNVkF+LgbuWBc74z5/C++mOinnqJpxw7yr1tCW3W1kxWefDjzEmkakCel3C+ltAKrgItP0H4R8C6AlHKvlDK3/X4xUA6EOlGrjgPosPeeqZHF3W44Yu+tifqHmgLY/yNMuAo0nvbZ0NLGZ9tKuHB8FD59MObzm3susc89S8u+fRxavJjW8nInqjz5cOanKhoo6PS4sP3YrxBCDAMSgO+7eW4a4A7sc4JGHQdi+SUNj9GjcAsOVltKv8ksySTSJ5JYXw0YxG1dBUhIXaS2kgHzxbYSGq02Fkzte32O+fTTif3Pf2gtLuHQNYtpLerdPtc6PdOrACGE6I+xfHeT0Mcrg1wIvCeltB3zupHAW8D1Ukp7N7qWCyGyhBBZejGcutgaLDRu3arp9Fab3caG0g1Mj5zu+msoUirWGvGnQWC82moGzOqsAhJDfZgU1z/vLp8Z0xn26gpsNTUcvGYx1oMHHSvwJKW3I4gXhRAbhBC39GEtoBDofBkWAxQfp+1C2qeXjiCE8AM+B/4ipczo7iQp5UtSyilSyimhofoMlJo0btwAra2aTm/dU7WHOmudNqaXDqVD9QHNG/MB5JU3sOlQtVI5PYDA7JWayrA3Xkc2N3PwmsU0793rQJUnJ70KEFLKU4GrUb7ws4QQ7wghzunhtI1AshAiQQjhjhIEPjm2kRAiBQgE1nc65g58CLwppVzbq3eioyqWtHSEpydekyapLaXfZJQo1yGaWKDOXgnuvjDqIrWVDJi1WQUYDYJLJw3c/sVz1CiGvf0WwmAgf/G1NO3Y6QCFJy+9XoNoXzT+C3APcDrwjBBijxCi2+ocKWUbcBvwFbAbWCOl3CmEeEgIMb9T00XAKtnVhWsBMBtY0ikNVrvGPicBlvR0vKdM0XTRUmZJJkkBSYR4hagt5cS0NMDOj2DMJeDuo7aaAdFqs/P+5iLOGhlGqK9jPjseiYkMW/k2BrOZ/CVLaNy0ySH9noz0dg1ivBDiKZQv+rOAi6SUo9rvP3W886SUX0gpR0gpE6WUf28/dr+U8pNObR6UUt57zHlvSylNR1Jg22/Z/Xh/OoNAa0kJ1v37NZ3earVZ2VK+RRujh10fQatlSEwv/bCnnMMNLVzp4F3j3GNjlX2uQ0LIv+HGjgw7nb7R2xHEc8BmYIKU8lYp5WboSEH9i7PE6WiDjvRWDa8/bK3YSrOtmekRGggQ2e9AcJJizqdx1mQVEurrwRkpjl9DNEVGKvtcx8VRcNPN1H//g8NfY6jTY4BoL3grkFK+JaVsOvZ5KeVbTlGmoxksaWkYQ0PwGJGstpR+k1GSgVEYmRLh4tuLVu2HQ2mKMZ+rZ1r1QHldMz/klHPZpJguxnyOxC0khGFvvI7HyJEU/u531H3xhVNeZ6jS41+lPfU0uH3hWEenC9Jux5K+HrOG7TVACRBjQsbg6+6rtpQTk/0OCANM0H7twwdbFGO+K6Y4b28SAGNAAHGvvYp3aipFd91Nzfsf9HySDtB7N9dDQJoQ4hOgwxlLSvmkU1TpaIbmXbux1dRoev2hwdrAzsM7WTp2qdpSTozdBtnvKluK+kWprWZASClZs7GAqfEnNuZzFEazmdiXX6Lwttspue8+7I2NBC3W/hqOs+ntuK4Y+Ky9vW+nm85JzlF7jZkqK+k/WWVZ2KTN9esfDvwEdYVDx5jvsIUrHLw4fSIMXl7E/N8LmOecTdnf/87hl14etNfWKr0aQUgp/+ZsITraxJKWhkdKCm4aLlTMLMnEw+jBhLAJaks5MVtWgmcApMxTW8mAWb2xb8Z8jsLg7k7MU09R/Kc/U/Hkkxg8PQi69tpB1aAlehUghBChwP8AY4CO3e6llGc5SZeOBrA3NtK4eTNBixerLWVAZJRkMClsEh5GF67haKqBPZ/BxMVg8uy5vQvT0NLG59tLuKiPxnyOQphMRP3zH8iWFsoefQxjQAD+8+f3fOJJSG+nmFYCe1AM9f4GHESplNY5iWnMytK8vcbhpsPk1eS5fv3DjvehrXlIbCv6+bbidmM+9QwRhdFI1BP/wnvGDIr/9Gfqf/xRNS2uTG8DRLCUcgXQKqX8SUq5FHDxCVsdZ2NJS0O4u+M9ZbLaUvqNZuy9s1dC2GiImqi2kgGzJquw3ZhP3S1eDB4exDz3HJ6jRlF0x++VCx6dLvQ2QLS2/ywRQlwghJiIYr6ncxKj2GtMxuCp3SmPzJJMfN19GRk0Um0px6d8DxRtUhanNZxKDJBXXs+mQ9VcOXVgxnyOwmj2Ifal/2CKiqLgt7fQvGeP2pJcit4GiEeEEP7AH4G7gFeAO52mSsflaS0royU3T9PprVJKMksymRYxDaPBqLac45O9EgxuMF77O++uzSrEzSD4zUTXub50CwoibsUrGHx8yL/xRqz5+WpLchl66+b6mZSyVkq5Q0p5ppRycmc/JZ2TD0u6Yr6r5QBRWF9IsaXYtdcfbG2wbTUkzwWzdjPF4IgxX6FDjfkchSkqirgVr0BrG/nLbtB3pmunt1lMr9HNZj/taxE6JyGWtDSMwcF4jBihtpR+k1Gq2Hu79PpD3rfQUAYTtV/7oBjzWVkwiLUPfcEjMZHYl/7DoSXXU3Djcoa99SZGPz+1ZalKb6eYPkPZvOdz4DvAD2hwligd10ba7VjWr8dn1iyEhvdCzijOIMw7jHi/eLWlHJ/st8EnFJLPVVvJgFmTVeA0Yz5H4TV+vLLH9f79FPz2FuxNv7KfO6no7RTT+51uK1H2a+jPNqQ6Q4CWnBxslZWaTm+1SzsbSjcwI3KGSyyWdoulEnK+VNYejCa11QwIxZivwqnGfI7CZ9Ysov/1L5o2b6bo93ciW1t7PmmI0t+/VDIQ50ghOtrBkpYGaNvee2/1Xmpaalx7/WH7GrC3Dglrjfc3K8Z8C5xszOco/M6bS8QDD9Dw008U33cf0m5XW5Iq9HYNop6uaxClKDvL6ZyEWNLT8UhOwhQepraUfnOk/sGl93/YslKpewgfrbaSASGlZG2WYsw3fBCM+RxF4MIrsdXUUPH00xgDAgj/059cd7TpJHrrxaQb8+kAYG9upjFrE4GLtG03nVGSQYJ/AuE+4U5/reaGVozuBkzufUilLdkKZdth3hPOEzZIZLUb8/32jES1pfSZ4JuWY6uuouqNN3ELCiLk5pvVljSo9HYE8RvgeyllbfvjAOAMKeVHzhSn43o0Zm1CWq34nKLd6aVWWyubyjZxceLFTn+tlsZW3vmbki01+bx4xsyOws3Ui0CR/Q4Y3WHsZU5W6HzWtBvzzRtkYz5HIIQg7J572kcS/8YYEEDgwoVqyxo0ersG8cCR4AAgpawBHnCOJB1XxpKWhjCZ8J7i4juvnYBth7fR1NY0KOmtm786RFN9KwHh3vyyNpeV92ew8+cibLYTzGm3WWHbGhh5AXgHOV2jM+kw5pugjjGfIxAGA5GPPIL5jDMo/dtD1K1bp7akQaO3AaK7dtr8a+sMCEt6Ol6TJ2Pw9lZbSr/JLMnEIAxO3160vqqZrd8VMmJ6OJfeNZmL75yIOdCDH1fm8M6DmeRklmK3/6q8CPaug6YqSNX+hjauYMznCITJRPTTT+E1eRJF/3MPDb+kqS1pUOhtgMgSQjwphEgUQgwXQjwFbHKmMB3Xo7W8nJacHE1nL4ESIEYFjcLfw9+5r/PJfgCmzx8OQExKIJfePZkLbh2PycPIt6/tYvUjG9i/pQIpOwWKLSvBNwoSz3SqvsFg9cYCksLMTIxV15jPERg8PYl94QU8EhMp/N3vaNq6VW1JTqe3AeJ2wAqsBtYATcCtzhKl45o0rj9ir6HdANHY2si2im1OT2+tKKgnJ7OU8WfG4Bfs1XFcCEH8uBCu/PNUzr1hDHabZN1/tvPeP7LI31mJrCuBvG9gwkJwZX+oXpBXXs/m/BqunOIaxnyOwOjnR9zLL+EWEkLB8ptoyctTW5JT6W0WkwW418ladFwcS3o6xsBAPEeNUltKv9lUtok22eb09Yf1H+Th4e3G5POHdfu8MAiSp4STODGUnMwyNn52gE+f3UpkWAMz2lKIGgK1D2uOGPNNilZbikNxCw0lbsUrHLzqKvKX3UD8OysxRQ+t93iEXo0ghBDftGcuHXkcKIT4ynmydFwNKSUN6en4zJypbXuNkgzcDe5MDHPevgr5Oysp2F3N1HkJeHifuALaYDQwalYkV/9tBrOvTKa2ys6HVY/y6TsNlB+qc5pGZ9Nqs/PB5kLOHhVGiNm1jPkcgXtsLHGvvIK9qYn8ZTfQVlWltiSn0Nv/9JD2zCUApJTVQI9VUkKI84QQOUKIPCHEr0YgQoinhBDZ7be9QoiaTs9dJ4TIbb9d10udOk6iZW8utorDmp5eAmX9ITUsFU835+xhYbdL0j/Yh1+IJ2Nn9/6q0mgyMC6plGuCb2Tm9BrKDtWx9rEs1v1nO5XF2rM9+97FjfkcgWdKCrEv/h+tpaUU3LgcW4P2/k490dsAYRdCdFhrCCHi6cbdtTNCCCPwPHA+MBpYJIToUhIqpbxTSpkqpUwFngU+aD83CCWNdjowDXhACBHYS606TmAo2GtUNVeRU53j1PWHnIxSKosamHFJIkZTH0da2W9jcndj0lXncO0js5h6YQIFu6tY9fAGvnltJ7UVjc4R7QTWbCwgzNeD00e4rjGfI/CeNImYfz9Nc04Ohbfehr2lRW1JDqW3n+D7gF+EEG8JId4CfgL+1MM504A8KeV+KaUVWAWcqDJpEfBu+/25wDdSyqr20co3wHm91KrjBCzp6bgPH44pUnvFTkfYULoBwGkBos1qI/OT/YQN8yVpch9tSFqbYMcHMGo+ePji7uXGtAsTuPaRWUw8J479myt454FMfli5h4bqZqfodxRldc38kFPOZZNd35jPEZhPP52oxx6lMTOT4rvuQra1qS3JYfTWzfVLYAqQg5LJ9EeUTKYTEQ0UdHpc2H7sVwghhgEJwPd9OVcIsVwIkSWEyKqoqOjFO9HpD/aWFho3btT05kCgTC+ZTWbGBI9xSv9bvy/AUtPCrMuS+p61s/szaKn71b4PnmYTsy5N4ppHZjLmtCj2pJfw9l8z+GVtLo11Vgeqdxzvby7ELhnS00vH4n/RRYT/+c/Uf/MtJQ8+2DVtWcP01mrjBuAOlH2os4EZwHrgrBOd1s2x4/3WFgLvSSltfTlXSvkS8BLAlClThsZfxAVp2rwZ2dIyJNYfpkRMwc3g+BrPpnorm748RPz4EKJH9GM2NPttCIiDYad2+7SPvwezF6WQek4cG784yLbvC9j5SzETzoph4jlxPS6GDxaKMV8h0+KDSAjxUVvOoBJ07WJsNdUcfuH/cAsMJOyPf1Rb0oDp7fjvDmAqcEhKeSYwEejpkr0Q6HwJEQMUH6ftQo5OL/X1XB0nY0lLA5MJn6lT1ZbSb4oaiiioL3BaeuvGLw7SZrUz8zf9MKSrKYD9Pym23j1kiPmFeHH2taNY9MB04scFs2ndId76y3qy1h3E2qz+1MbGg9UcOGzRfOV0fwm5/XYCFi2k8uVXqFyxQm05A6a3l1LNUspmIQRCCA8p5R4hREoP52wEkoUQCUARShC46thG7f0EooxIjvAV8Ginhelz6XnNQ8dJNKSl452aisFHu1eEzrT3rilrZOdPRYw+JZKgyH78jra+C0iY0HuH3MAIH+beMJZJc+vZ8Ml+Mj/ez7bvC/pmCOgE1mQVYPZwY964CFVeX22EEET85S/Ya2sp/9cTGAMCCbjsUrVl9ZveBojC9jqIj4BvhBDV9HBFL6VsE0LchvJlbwRelVLuFEI8BGRJKT9pb7oIWCU7TdpJKauEEA+jBBmAh6SUQzPR2MVpq6ykZfduQn9/h9pSBkRGSQYhXiEkBjjecjrj430YTAamXpjQ95PtdsheCQmzIbD7oroTERrrywW3TqB0fy0ZH+/nl7W5ZH+bz5R58YycFYlxEBeJ65tb+XxbCZdMjMLb/eS1ahNGI1H/+Ae22jpK/vpXjP5++M6Zo7asftHbSurftN99UAjxA+APfNmL874Avjjm2P3HPH7wOOe+CrzaG306zsOSfsReQ7sL1FJKNpRsYHrkdIdbPpTur2Xf5gqmXpiAj38/CsLy06H6IJzx5wHpiBjuzyV3TqRwTxUZH+/nx5U5bP46n2kXJpA8NRyDwflWF59vK6Gp1cYVJ9Hi9PEQ7u7EPPsMh66/nqI//JHYl1/GZ/o0tWX1mT5fXkgpf5JSftKeuqozxLGkp2Pw98dztHZ3NcuryaOyudLh6w9SStLey8Pbz53UOf38UtyyEjz8YNRFDtEUMzKIy/5nMhfcMh6T+wkMAZ3A6qwCkoeIMZ8jMHh7E/vii5jiYim85Raadu5UW1KfGfpJyjr9RkqJJS1Nsdcwatc4LqNE2bDH0QHiQPZhSvfXMu2iBNw9+zGl0lIPuz6CMb8Bd8fZpwshiB8fwpX3HccQ0AmBIresni35NSwYQsZ8jsAtMJC4FSsw+vtTcONyWg4cUFtSn9ADhM5xse7bR1t5OT6zZqotZUBklmQS5xtHpNlxRX42m530D/MIjPBm1Kx+9rvrY2htVLKXnMARQ8BF90/jrGtH0lTfyqfPbuWjJ7dQnFfTcwd9YE1WwZA05nMEpvBwYle8AkDBshtoLStTWVHv0QOEznE5aq+h3fWHNnsbWWVZDq+e3vVzMbXlTcy8NAlDfxeCt6yE4GSIde7ctGIIGMXVf5vBaVeOoKaskQ+f2Mynz2Y7xBBQMeYrGrLGfI7AIyGB2JdewlZbS/6yZdhqHBugnYUeIHSOS0N6Ou7x8bjHaPeqcMfhHVhaLQ4NENamNjZ+foCo5ADixwX3r5PKfcoCdepVMEhTMkaTgfFnxnDNIzOZ+ZtEyg46xhDwu93lVFqsXHmS1j70Fq+xY4h54QVa8wsouOlm7I2u762lBwidbrFbrTRu2Khpcz5QppcEgmkRjrtK3/y1ss90vyw1jpD9DghDn2ofHIXJ3cikucNY/Mgspl4QP2BDwLVZijHf7OShbcznCHymTyP6yf+laft2Cn93B9Lq2rk+eoDQ6ZamzVuQTU34nKrd6SWAzNJMRgaNJNDTMWbADdUtbP22gOQpYYTH+/WvE7tNKY5LPBv81DM/9PByY9pFw1n8yEwmzoljX7sh4I99MAQ8Ysx3+UlizOcIfOfMIfLhh7D88gvF996LtNl6PkklTt5qFp0TYklPB6MR72nay90+QlNbE9nl2Vw9ynGLwBs+3Y/dLplxyQAK7vb/CHVFMPfvDtM1ELzM7sy6LIkJZ8eSte4gu34pZs/6UsaeHs2kucPw9nM/7rlHjPn02oe+EXDZZdhqatqrrQMI/+tfXTL7Sw8QOt1iSUvDKzUVo9mstpR+s6VsC632VoetP1QWNbBnfQnjz4rFL8Sr5xOOR/ZK8AqElHkO0eUofAI8OH1RChPPiWPj5wd6NATsMOZLOPmM+RxB8LJl2KqrqXxlBcaAQEJ/d7vakn6FPibU+RVt1dU079ql+fTWjNIM3AxuTAqb5JD+0j/Yh7uXG1Pmxfe/k6Zqxdp73BXg5poZP34hXpx93WgWPTCdYWOObwjYYcynjx76Tegf/4j/5Zdx+IUXqHrrbbXl/Ap9BKHzKxrXrwcpMWvYXgOUBerxIePxNg28CK1gTxX5OyuZeWkinj4DsNbe8T7YWpxW++BIAiN8OG/5WCry68n8VDEEzP42H3OAsl1rcW0T1zd7Yv28iFVf9GC23I/Zk8GYcXGJaR2v+VjPmIj9y3pM27/BGND3SvSgKB/mLHG824EeIHR+RUN6OgZfXzzHjlVbSr+pballd+Vufpv62wH3Je2S9Pfz8A3yZPyZMQPrLPsdCB8LkRMGrGuwCI3z5cJbJ1Cyr5Yd/y2ktdlGm11SWFFHVJAnAaEDmG7rhkHba8eFNvXx9htO07Zt2A7uxmPsWNyC+5Y+7WV2zn4geoDQ6YJir5GOz4wZCDftfjw2lm5EIh1ir7F3YxmHCxqYc/3ogdlol++Bok0w99FBq31wJJGJ/kQm+gPw7oZ8Pigo5MMbJjMxTt8u3hHYGpLJv/Y6Wj54jbhXV+A9yTFTowNBX4PQ6YL1wAHaSko07d4Kiv+St5s3Y0MGNgpqa7WR8fE+QuN8GTE1fGCist8GgxuMv3Jg/bgAa9qN+VJ1Yz6HYTSbiX35JUwRERTc/Fuac/aqLUkPEDpdsaSl9PSLNgAAGpVJREFUAwyJ7UUnh0/GZBjY0HvbD4U0VLUw69JExEAss22tsHU1jDgPfEIGpEltjhjzXTlVN+ZzNG7BwcSteAWDlxcFN9yAtaBAVT16gNDpgiUtDVNcHO6x2s1MKbWUcrDu4IDTW5sbWtm07hBxY4KJGRk0MFF534KlXBOL0z1xxJjvkonatWBxZUzR0cSteAVptZK/7AbaKnra3dl56AFCpwNptdK4YYPm01uPbC860PWHrHUHaW1uY9alDtiFbsvb4BMKyecMvC8VsbYpxnxzRoXrxnxOxCMpidiX/kPb4cPk37gcW93ATRX7gx4gdDpo2roVe2Oj5tcfMksyCfIMIjkwud991FY0sf3HQkbOiiQ4eoDFgpbDsPdLZe3B6Jxsk8Hi+z2KMd+CqQPM5tLpEa8JE4h55hla9u2j4JZbsDf3zv7EkegBQqeDhvR0MBjwme5Ya+zBREpJRkkG0yKmYRD9/3hnfrwPg0Ew7cLhAxe1bQ3Y22DiNQPvS2XWZBUQ7qcb8w0W5lNPIfrxf9K0aTNFd/4B2do6qK+vBwidDixp6XiNH4/Rr58mdC7AgdoDVDRVDGj9oexgHblZ5aSeE4c5cIDTKFYLZL0KUZMgbNTA+lKZnNJ6ftSN+QYdv/PPJ+L+v9Lwww+U/OWvSLt90F5bu4nuOg7FVlND844dhNx8s9pSBsSR7UX7GyCkVIrivHxNTDwnbmBibG2wdglU7YOr1gysL5UpqW1iyWsbCDF7cO3MeLXlnHQELlqEraaGin8/gzEggLB77xmUDDI9QOgAYMnIBLtd+/beJZlEm6OJ9e1fFtbB7ZUU59Ywe+EI3L0G8O8hJXx+J+R+DRc+penF6dqmVpa8upH65jbW3DSTcD9PtSWdlATffDNtVdVUvfEGxqAgQm5a7vTX1AOEDqCktxrMZrzGjVNbSr+x2W1sLNvIucPO7df5dpud9R/kERDuzejTogYm5qd/wuY3YfbdMGXpwPpSkZY2Gze9lcX+ww28fv00Rkdpd/pR6wghCP/TvcpI4qmnMAYEEHjlAqe+ph4gdNrtNdLwnj4dYdJuls3uqt3UW+v7Pb20O72E6tJGzr9pHMaBzLFvfhN+fEypeTjzvv73ozJ2u+SutdvI2F/F01emckqStgv8hgLCYCDq0b9jr6uj9MEHMfr743feXKe9nr7SpEProUO0Fhdrvnr6yPpDf7YXtTa3kfnpASIT/UlIHcAX4d6v4NPfK7vFXfRvTXouHeGxdbv5dGsx954/Ui+KcyGEyUT000/hNXEixXffrWzu5SScGiCEEOcJIXKEEHlCiHuP02aBEGKXEGKnEOKdTscfbz+2WwjxjNBr+p1GQ/sHzKzx/aczSjJIDkwm2KtvTpgA2d8W0FRnHdg+00WblEXpiHGw4E1N1zys+OUAL/98gCWz4rlptgNSfXUcisHLi9gX/w/3hAQKbrudpm3bnPM6TukVEEIYgeeB84HRwCIhxOhj2iQDfwJOkVKOAX7ffnwWcAowHhgLTAVOd5bWkx1LWjqm6GhMw4apLaXftNhayC7PZnpE36eXLLUtbPkmn8RJoUQM9++fgMp9sHKBUi199Vrw0O5OfJ9tK+aRz3dx3piI/2/vzqOjrO89jr+/WUnCDmEHQRbFDYQQBNyvbVEsVi0ut66cav2j57ZX7aLdzm2vve21tr3nnt5z3HBfinVDwaq1uCVkQsCELURDUAgkEMKaSUImme/943miQ5iEZJJh5pn5vs6Zc2Yenmf4PofwfPMsn9+PX1x5ho23FKdSBw5k/GOPkjZ0KLvvvz8qc1tH8x5EPlCpqlUAIvIicBWwJWSdO4C/qOoBAFXd6y5XoB+QgTPVSDqwJ4q1Ji0NBGj0+Rh4+eWePhCU7i3laNtR5o3p+TAhxW9uJxgIRj7PdEMdPHstaBBuegX6j4jse+JAUVU9d/+1jLxThvDnG2aS2psBCk3UpY8YwYRlj0MwiKT2Yij6TkTzEtNYIHQowmp3WahpwDQRKRCRIhFZCKCqa4DVQI37eltVyzv+BSJyp4iUiEhJXQwHtPKypo0bCTY0JMTwGmmSxuyRs3u03f4aP+Uf7+bMi8YyeEQEM8+1+OH56+BIrZN1GD6l598RJypqj3DH0yVMGJbNo7fk0a83c1+YkyZjwgQyJk6MyndHs0GE+9Wj4xROacBU4GLgRuAxERksIlOA6cA4nKZyqYhceNyXqT6iqnmqmpeba9H/SPgLCkGEnPO8O7wGOA3irOFnkZOe06Pt1ry6jfTMVOZEMs90exCuphSWPAHj5/T8O+JEexAuKz2Vp5bmMzg7I9YlmTgQzQZRDYSmlcYBHSeurQZeV9WAqm4HKnAaxtVAkao2qGoD8BbQ+6nBzHH8BQX0O/vsiObBjReHWw6zqX5Tjx9v3fXpAT7fsI9ZC08ha0APD4iq8OYPnSDcoofgtMt7tn0cCQ3CPXl7PmMH9+0Uosa7otkg1gJTRWSSiGQANwArOqzzGnAJgIgMx7nkVAXsAC4SkTQRSce5QX3cJSbTO22HD9O0YYPnH28tqS0hqMEeNYj2eab7D8lkxqURpK7f/x188gxc+GPPB+HufNoJwj1882wLwpljRK1BqGor8H3gbZyD+3JV3SwivxaRxe5qbwP1IrIF557Dj1S1HvgbsA3YCJQBZar6RrRqTVZ+nzO8htcfb/XV+MhKy2JG7oxub1O5bi97vzjC3MWnkpbRw2vt656CD34HM2+CS+7vYbXxIxhU7llehm/7fv6wZIYF4cxxopqkVtVVwKoOy34Z8l6Bu91X6DptwPeiWZtxh9fIziZrRvcPrPHIV+Nj1ohZZKR27zJRWyBI0evbGDa2P9PmjurZX/bp2/Dmv8OUy+Cbf/Z0EO63q8p5c0MN911+OlfNtCCcOZ4lqZOYv3AN2fn5SIZ3b0jWNdax7dC2Hl1e2vhBNYf3NTP/2smk9OQxzuqQINySpzwdhHvsoyoe+9gJwt1pQTjTCWsQSapl504CO3Z4/vHWng7v3ewPULLqc8ZPH8KEM3qQuK7fBs8vSYgg3Btlu/nPleVcfpYF4UzXrEEkKX+BM7yG129Q+2p8DMocxOlDT+/W+uv//gVHm1qZf20P8grtQTiAm1/1dBBuzbZ67lleRv7EofzpegvCma7ZaK5Jyl9QQNro0WRMmhTrUiKmqvhqfd2eXvRwfRMbVldz+txRDB83oHt/ydEG58zhSC3c9iYMizBtHQcqao9w5zNOEO6RW2ZbEM6ckJ1BJCFtbcVfVETO/Hmevryw48gOav21nDe6exEZ34oqEMhf3M1r7m0BNwhX5gThxuVFXmyM7T7YxK3LisnOsCCc6T5rEEmoedMmgkeO0N/j9x98NT6ge/cf6nYc4VPfHmZcOp4BQ7sxI1p7EK7yXVj0R+8H4Z4oxn/UgnCmZ+wSUxJqKCgAEbLn9Xxgu3hSVFPEqJxRTBjQ9dzRqkrBy5X0y0ln1sJujlj7/u/gk2fdINztfVBtbLQH4bbv8/PU7flMH21BONN9dgaRhPyFa+h3xhmkDRkS61IiFtQgxbXFzB0194SXyXZs3s+uigPkLZpIZnfmmV73ZMIE4e4OCcLNtyCc6SFrEEmmraGBptJSzz/eunX/Vg4dPXTCy0vBoFL4SiUDc7M468JuhMEq/u4G4b7m+SDcA6vKWbmhhvuvsCCciYw1iCTTWFwMbW3kJMDwGsAJb1BvXVPD/t1+5n1rMqlpJ/hxry5xg3DnwJInPR+Ee9wNwt1xgQXhTGSsQSQZ/8cFSFYWWbPOjXUpveKr8TF50GRyszsf5j1wtI3iFVWMnDSQybNOMBx8/TZnXocBIxMmCHfF2RaEM71jDSLJ+AsLyZ6TR4qHh9cItAVYv3f9CS8vlb23E/+hbswz3bAXnr3Gee/xGeFCg3B/vM6CcKZ3rEEkkZbqXbR8/rnnH28tqyujqbWpywbReLiF9e98waQZwxkzpYu5Lo42uDPC7XFmhPNwEG5r7WHufKaEU2xGONNH7DHXJOIvLADw/P2HopoiUiSFvFGdB9dKVm6ntSXIvKu7OOCHBuFueMHzQbjblq0lOyOVJ5fmMyjbu/dPTPywBpFE/IVrSBsxgowp3p03GZz7D2cOO5OBGeGf6T+4p5HNH+3mzPPHMGRUJ1OQhgbhvvk/cNrCKFYcXaFBuOV3zbMgnOkzdokpSWhbG/41a8hZsMDTNy39AT+b9m3q8umlNa9uIzU9hTlXdjHO1Pv/5QThLvoJzL6t7ws9SZoDXwXhHr55tgXhTJ+yBpEkmrdsIXjokOcvL63bs45Wbe30/kNN5UGqSus49+sTyB7YyY34kifgg9/DuTfBxfdFsdroCgaVe16yIJyJHmsQScJf0H7/wfvDa2SmZjJzxMzj/kzVCcVlD8pg5mWdDL9R8XdYebcThLvSgnDGdMUaRJLwFxSSOX06acN6MElOHCqqKWLmiJlkpmYe92dVn9RRW3WYuYtPJT0zzBM87UG40TMSJgh3+wILwpnosQaRBIJ+P42lpfT3+ORA9U31fHbgs7D3H9pag6x5dRtDx+Rw+rzRYTZuD8KNgn/1dhBuhRuEW3T2aH6xyIJwJnqsQSQB/9q1EAh4/v5DcW0xEH54jc0f7eZQXRPzrg4zz/QxQbiXof8JUtVxrHDbPu5dXkb+pKE8dN2Mns2pbUwP2WOuScBfUIhkZpI1e3asS+kVX42PARkDmD50+jHLjza1snbldsaeNphTzupwCe1oAzy3xGkSt3p7RrittYf53tPrnCDczRaEM9FnDSIJ+AsKyM7LIyXz+Ov2XlJUU8SckXNITTn2wLj+7S9obggw/5oOQ2q0BeClW6F2gxuE826D/DIIl+nMCGdBOHMy2CWmBBeoqaGlqsrzw3vvPLKTXQ27jnu8teFAM2Xv7WRa/khGnBKSAVCFN34Ilf+AK//k7SBc47Ezwo2xIJw5SewMIsH5CwsBPN8gOhve27eiClVlbsd5plf/FkqfhYt+6vkg3B3PuDPCLbUZ4czJFdUzCBFZKCIVIlIpIj/tZJ3rRGSLiGwWkedDlk8QkXdEpNz984nRrDVR+QsKSc0dTua0qbEupVd8NT5ys3KZNOirdPS+6ga2FtVyziXjGTg85Lfqkifgw/+Gc2+Gi8P+2HlCMKjcs7yM4u37eei6mcyfbEE4c3JF7QxCRFKBvwBfA6qBtSKyQlW3hKwzFbgPWKCqB0QkdJzlp4EHVPVdEekPBKNVa6LSYBD/mjX0v/ACTz8K2T696IIxxw4TsuaVSjKz0pgdOs90xVtOEG7q151LSx7db1XlNyu3sHJjDT+7YjqLZ4yJdUkmCUXzDCIfqFTVKlVtAV4Eruqwzh3AX1T1AICq7gUQkTOANFV9113eoKqNUaw1ITWXl9N24IDnH2/97MBn7G/ef8z9h51b9rNjy37yrphIvxz3hu3OtfDS7TB6ZgIE4bbzRMHnLF0wie9e0MWYUsZEUTQbxFhgZ8jnandZqGnANBEpEJEiEVkYsvygiLwiIp+IyIPuGckxROROESkRkZK6urqo7ISX+Quc+w/Z87w9vEb7/Yf2BqFBpeCVSgYM68fZF41zVtpXCS9c7wbhlkNGJ6O4esCKst08sMoJwv180XRPn/0Zb4tmgwj3U60dPqcBU4GLgRuBx0RksLv8AuBeYA5wKnDbcV+m+oiq5qlqXm6ud8NP0eIvKCBz2jTSR3h3hjQAX62PiQMnMipnFAAVxbXUVzdw3rdOJTU9JSQIJ94PwlXu457lpRaEM3Ehmg2iGhgf8nkcsDvMOq+rakBVtwMVOA2jGvjEvTzVCrwGzIpirQkn2NhI0/r1nn96KRAMUFJb8uXZQ2tLG77XqxhxygCmzh75VRDOX+f5GeHKaw7zvWfWMWl4jgXhTFyIZoNYC0wVkUkikgHcAKzosM5rwCUAIjIc59JSlbvtEBFp/1XwUmALptsaS0rQBBheY9O+TTS2Nn75eOuG1dU0HDjqhOK01Q3CbXTuOXg4CLfrYBO3PVFMTmYaT95uQTgTH6LWINzf/L8PvA2UA8tVdbOI/FpEFrurvQ3Ui8gWYDXwI1WtV9U2nMtL74nIRpzLVY9Gq9ZE5C8oRDIyyM7z7kETnPS0IMwZNYemhhbWvfU5E88exthpg+GNH3wVhJv2jViXGrFDjQFuW1ZM49E2nlw6x4JwJm5ENSinqquAVR2W/TLkvQJ3u6+O274LnBPN+hKZv7CA7LzZpGR5+2Djq/Exfdh0BmUO4qPXPyVwtI15V0+B1Q9A6XPOhD+zb411mRFrDrRxx9MlfFHfyJNL53D6KAvCmfhhQ20koMCevRz9rNLzl5caA42U1ZUxd/RcDtU1sumDXUxfMIahu/8KHz4Is25xpgz1qGBQuXt5KcWf7+cP182wIJyJO9YgElCiDK+xfu96WoOtnDfqPIpeqyIlVcifXgUr73GCcIu8H4RbtbGWny+yIJyJT9YgEpC/sJDUoUPJPO20WJfSK74aH+kp6YxtmkLlur3MzE8j562lIUE47w4ldmwQzmaEM/HJGkSC0WAQf2EhOfPnIyne/uf11fiYmTuTda/tJKt/Cufu/G5CBOFeL93lBOHOcYJwxsQrbx9BzHGOVlTQVl/v+fsPB5sPsnX/VuY0X0pN5SHyc14kIzWQEEG4e18qY+6koTy0xIJwJr559xzdhPXV/QdvN4ji2mIkKGSuHUu/zL2ckf4GfGdFwgThHrnFgnAm/lmDSDD+ggIyp04hfeTIqP9dQQ3S3NpMU2tTt16NgcZur9vU2sQ59RfRVK9cMuRxUq5bBmO9m+k4LgiXZUE4E/+sQSSQYHMzjSXrGHLjDV8uawu2hT9Yt3Y4WAe6f+AOffVEiqSQlZYV9jW039BjlwVzyFw7kdz0zUz89k2eD8LduqyYxpY2XrprngXhjGeIk1Xzvry8PC0pKenxdpVlJbz/p4ooVBQDCmlBqB8IzRnOyIg9/deV9pc6N6gk5JWi4T+ndLVdh8/d1aYZNLUNI2vUP3hhoHebA8DBpgCHGgM8tTSfeZOHxbocY44hIutUNS/cnyX9GURaZj9Ea2NdRp9pSRNaclJJFSEVSEFIUY55n4KE/6zSo4N4tGUMrKVw8jVM9WjWoZ0gfDtvnDUH4zlJfwZhjDHJrKszCHvM1RhjTFjWIIwxxoRlDcIYY0xY1iCMMcaEZQ3CGGNMWNYgjDHGhGUNwhhjTFjWIIwxxoSVMEE5EakDvujFVwwH9vVRObGUKPsBti/xKlH2JVH2A3q3L6eoatgx9BOmQfSWiJR0lib0kkTZD7B9iVeJsi+Jsh8QvX2xS0zGGGPCsgZhjDEmLGsQX3kk1gX0kUTZD7B9iVeJsi+Jsh8QpX2xexDGGGPCsjMIY4wxYVmDMMYYE5Y1CJeI/EZENohIqYi8IyJjYl1TpETkQRHZ6u7PqyIyONY1RUpElojIZhEJiojnHkkUkYUiUiEilSLy01jX0xsiskxE9orIpljX0hsiMl5EVotIufuz9YNY1xQpEeknIsUiUubuy3/06ffbPQiHiAxU1cPu+38DzlDVu2JcVkRE5OvAP1W1VUR+D6CqP4lxWRERkelAEHgYuFdVPTNtoIikAp8CXwOqgbXAjaq6JaaFRUhELgQagKdV9axY1xMpERkNjFbV9SIyAFgHfMuL/y4iIkCOqjaISDrwMfADVS3qi++3MwhXe3Nw5QCe7Zyq+o6qtrofi4BxsaynN1S1XFUrYl1HhPKBSlWtUtUW4EXgqhjXFDFV/RDYH+s6ektVa1R1vfv+CFAOjI1tVZFRR4P7Md199dmxyxpECBF5QER2At8BfhnrevrIUuCtWBeRpMYCO0M+V+PRA1GiEpGJwLmAL7aVRE5EUkWkFNgLvKuqfbYvSdUgROQfIrIpzOsqAFX9maqOB54Dvh/bart2on1x1/kZ0IqzP3GrO/viURJmmWfPTBONiPQHXgZ+2OEKgqeoapuqzsS5UpAvIn12+S+tr77IC1T1sm6u+jywEvhVFMvplRPti4jcClwJ/IvG+Y2mHvy7eE01MD7k8zhgd4xqMSHc6/UvA8+p6iuxrqcvqOpBEXkfWAj0yYMESXUG0RURmRrycTGwNVa19JaILAR+AixW1cZY15PE1gJTRWSSiGQANwArYlxT0nNv7D4OlKvqH2NdT2+ISG77U4oikgVcRh8eu+wpJpeIvAychvPEzBfAXaq6K7ZVRUZEKoFMoN5dVOThJ7KuBv4XyAUOAqWq+o3YVtV9InIF8GcgFVimqg/EuKSIicgLwMU4Q0vvAX6lqo/HtKgIiMj5wEfARpz/7wD3q+qq2FUVGRE5B3gK5+crBViuqr/us++3BmGMMSYcu8RkjDEmLGsQxhhjwrIGYYwxJixrEMYYY8KyBmGMMSYsaxDG9ICINJx4rS63/5uInOq+7y8iD4vINnckzg9FZK6IZLjvkyrIauKPNQhjThIRORNIVdUqd9FjOIPfTVXVM4HbgOHuwH7vAdfHpFBjXNYgjImAOB50x4zaKCLXu8tTROT/3DOCN0VklYh8293sO8Dr7nqTgbnAz1U1COCO+rrSXfc1d31jYsZOYY2JzDXATGAGTrJ4rYh8CCwAJgJnAyNwhpJe5m6zAHjBfX8mTiq8rZPv3wTMiUrlxnSTnUEYE5nzgRfckTT3AB/gHNDPB15S1aCq1gKrQ7YZDdR158vdxtHiTmhjTExYgzAmMuGG8u5qOUAT0M99vxmYISJd/R/MBJojqM2YPmENwpjIfAhc707WkgtcCBTjTPl4rXsvYiTO4HbtyoEpAKq6DSgB/sMdXRQRmdo+B4aIDAPqVDVwsnbImI6sQRgTmVeBDUAZ8E/gx+4lpZdx5oHYhDOPtg845G6zkmMbxneBUUCliGwEHuWr+SIuATw3uqhJLDaaqzF9TET6u5PID8M5q1igqrXueP2r3c+d3Zxu/45XgPs8PB+3SQD2FJMxfe9NdxKXDOA37pkFqtokIr/CmZd6R2cbu5MLvWbNwcSanUEYY4wJy+5BGGOMCcsahDHGmLCsQRhjjAnLGoQxxpiwrEEYY4wJ6/8B8FVw69M9tlYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(grid.best_score_)\n",
    "print(grid.best_params_)\n",
    "\n",
    "# plot CV误差曲线\n",
    "test_means = grid.cv_results_[ 'mean_test_score' ]\n",
    "\n",
    "# plot results\n",
    "n_Cs = len(Cs)\n",
    "number_gamms = len(gammas)\n",
    "\n",
    "test_scores =  np.array(test_means).reshape(n_Cs,number_gamms)\n",
    "\n",
    "x_axis = np.log10(Cs)\n",
    "for i, value in enumerate(gammas):\n",
    "    pyplot.plot(x_axis, test_scores[:,i], label= gammas[i])\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'log(C)' )                                                                                                      \n",
    "pyplot.ylabel( 'accuary' )\n",
    "pyplot.savefig('SVMGridSearchCV_C.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "这个例子中，C越大，最佳的gamma越小\n",
    "\n",
    "参数C：\n",
    "当C很小时，模型出现欠拟合交叉验证得到的准确率很低\n",
    "除了gamma取值非常小的情况外，随着C的增大，模型越来越复杂，交叉验证得到的准确率是线增大后减小的，开始减小说明C太大，出现了过拟合\n",
    "\n",
    "参数gamma:\n",
    "当gamma设得合适时，测试准确率受C的影响很大   \n",
    "当gamma设的太大，不管C的取值是多少，测试准确率都不高，发生过拟合\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
